curl – A Nifty Command Line Download Tool For Linux

curl is a tool to transfer data from a server or to server, using one of the supported protocols (DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and TFTP).

The command is designed to work without user interaction.

Also curl support proxy support, user authentication, FTP upload, HTTP post, SSL connections, cookies, file transfer resume, Metalink, and more.

curl is powered by libcurl for all transfer-related features.

If you specify URL without protocol:// prefix, curl will attempt to guess what protocol you might want.

For example, host names starting with "ftp."curl will assume you want to speak FTP.

If it’s not find specific protocol, then do default to HTTP. Alternatively you can install other Download Utilities too.

1) Install curl on Linux

We can easily install curl command line download tool to all the Linux Distribution such as Debian, Ubuntu, Mint, RHEL, CentOS, Fedora, suse, openSUSE, Arch Linux, Manjaro, Mageia, etc.. Just fire the below command to install.

For Fedora system, use DNF Command to install curl.

$ sudo dnf install curl

For Debian/Ubuntu systems, use APT-GET Command or APT Command to install curl.

$ sudo apt install curl

For Arch Linux based systems, use Pacman Command to install curl.

$ sudo pacman -S curl

For RHEL/CentOS systems, use YUM Command to install curl.

$ sudo yum install curl

For openSUSE Leap system, use Zypper Command to install curl.

$ sudo zypper install curl

2) Download Single File

The below command will download the file from given URL and stores in current directory, while downloading the file we can see the (total time, time left, Avg download speed, Current download speed & download progress) of file.

# curl -O https://curl.haxx.se/download/curl-7.48.0.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 8579k  100 8579k    0     0  3532k      0  0:00:02  0:00:02 --:--:-- 3533k

3) Save the file with different name

We can save the file with different name & format while initiate downloading, using -o (lowercase) option. Here we are going to save the filename with curl.zip.

# curl -o curl.zip https://curl.haxx.se/download/curl-7.48.0.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 8579k  100 8579k    0     0  3605k      0  0:00:02  0:00:02 --:--:-- 3606k

4) Download Multiple Files

The below command will download more then on file from the location and stores in current directory, while downloading the file we can see the (total time, time left, Avg download speed, Current download speed & download progress) of file. We need to mention -O (uppercase) to each URL.

# curl -O https://curl.haxx.se/download/curl-7.48.0.tar.gz -O https://download.owncloud.org/community/owncloud-9.0.0.tar.bz2
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 8579k  100 8579k    0     0   463k      0  0:00:18  0:00:18 --:--:-- 1145k
100 21.6M  100 21.6M    0     0  5116k      0  0:00:04  0:00:04 --:--:-- 5270k

5) Resume Incomplete download

Continue/Resume a previous file transfer at the given offset. The given offset is the exact number of bytes that will be skipped, counting from the beginning of the source file before it is transferred to the destination. Use -C - to tell curl to automatically find out where/how to resume the transfer. It then uses the given output/input files to figure that out.

# curl -O https://download.owncloud.org/community/owncloud-9.0.0.tar.bz2
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0 21.6M    0 16384    0     0  23752      0  0:15:54 --:--:--  0:15:54 23744^C


# curl -C- -O https://download.owncloud.org/community/owncloud-9.0.0.tar.bz2
** Resuming transfer from byte position 573440
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 21.0M  100 21.0M    0     0  5687k      0  0:00:03  0:00:03 --:--:-- 5688k

6) Limit download speed

By default curl utilize full bandwidth for downloading file and we can’t use anything on server before download completion (Which will affect other service accessing bandwidth). So better use --limit-rate option to avoid further issue while downloading big size file.

# curl --limit-rate 500k -O https://curl.haxx.se/download/curl-7.48.0.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 8579k  100 8579k    0     0   499k      0  0:00:17  0:00:17 --:--:--  496k

7) Download metalink file

This option can tell curl to parse and process a given URI as Metalink file. It will also verify the hash of the file after the download completes. The Metalink file itself is downloaded and processed in memory and not stored in the local file system.

# curl --metalink http://releases.ubuntu.com/16.04/ubuntu-16.04-desktop-amd64.metalink

8) Get Response Header info

(HTTP/FTP/FILE) Fetch the HTTP-header only! HTTP-servers feature the command HEAD which this uses to get nothing but the header of a document. When used on an FTP or FILE file, curl displays the file size and last modification time only.

# curl -I https://www.2daygeek.com
HTTP/1.1 200 OK
Date: Thu, 05 May 2016 11:55:33 GMT
Set-Cookie: __cfduid=da343c5351ff9781b868922f916e36de51462449331; expires=Fri, 05-May-17 11:55:31 GMT; path=/; domain=.2daygeek.com; HttpOnly
Server: cloudflare-nginx
CF-RAY: 29e401004de72dd3-BOM
Connection: keep-alive

9) Download a file from password protected site

Alternatively we can download a file from password protected site. The below command will download the file from password protected site.

# curl -u username:password -O https://download.owncloud.org/community/owncloud-9.0.0.tar.bz2

# curl -u username:password -O ftp://ftp.gnu.org/gnu/wget/wget-1.17.tar.gz

10) Upload a file to ftp server

Curl can also be used to upload files to the FTP server by adding -T option

# curl -u username:password -T myfiles.zip ftp://ftp.2daygeek.com

# curl -u username:password -T "{myfile1.zip,file2.zip}" ftp://ftp.2daygeek.com

11) Download a file using proxy

Alternatively we can download a file using prxy by adding -x option with curl.

# curl -x http://proxyserver:port --proxy-user username:password https://www.2daygeek.com

12) Read more about curl

If you want to know more option which is available for curl, you can grep the details on your terminal itself by firing below commands..

# man curl
or
# curl --help

Enjoy…)

About Magesh Maruthamuthu

Love to play with all Linux distribution

View all posts by Magesh Maruthamuthu

Leave a Reply

Your email address will not be published. Required fields are marked *