The curl command in Linux: transfer data from URLs

Introduction

In the world of system administration and software development, transferring data to or from remote servers is a daily task. One of the most versatile and lightweight tools to achieve this in Linux environments is the curl command. Although its name comes from ‘Client URL’, its functionality goes far beyond simple file download, allowing interaction with APIs, testing endpoints, and automating workflows without needing graphical interfaces.

What is curl?

Curl is a command-line tool and a libcurl library that enables data transfer using various protocols such as HTTP, HTTPS, FTP, FTPS, SCP, SFTP, IMAP, POP3, and SMTP. Its design focuses on simplicity and power, offering a straightforward syntax that can be integrated into shell scripts, cron jobs, or CI/CD pipelines. Thanks to its open-source license, it is available on practically all Linux distributions.

Installation

In most modern distributions, curl comes preinstalled. If it is not present, it can be added via the corresponding package manager. On Debian or Ubuntu run sudo apt update && sudo apt install curl. On Red Hat, CentOS, or Fedora use sudo dnf install curl or sudo yum install curl depending on the version. On Arch Linux the command is sudo pacman -S curl. After installation, simply run curl –version to verify.

Basic Syntax

The simplest way to use curl is to provide a URL as an argument. For example, curl https://example.com returns the resource’s content to standard output. To save the result to a file, add the -o option followed by the filename, as in curl -o file.html https://example.com. You can also use -O (uppercase) to have curl retain the original name of the remote resource.

Usage Examples: File Download

Suppose we want to obtain the latest version of a compressed package from a repository. The command curl -L -o latest.tar.gz https://example.com/packages/latest.tar.gz follows redirects thanks to -L and saves the file with the indicated name. If we only need to view the server headers, curl -I https://example.com shows information such as the status code, content type, and last modification date.

Sending Data: POST and PUT Requests

To interact with REST APIs it is common to send data in the request body. With curl this can be done using -X POST or -X PUT together with -d to specify the data. For example, curl -X POST -d ‘name=juan&age=30’ https://api.example.com/users sends a form encoded as application/x-www-form-urlencoded. If JSON is preferred, add -H ‘Content-Type: application/json’ and pass the object with -d ‘{name:juan,age:30}’.

Useful Options

Some flags that are very handy in daily use include -v or –verbose to see the full exchange of headers and bodies, -s or –silent to suppress the progress meter and error messages, -S or –show-error to show errors even when using -s, -L or –follow-location to follow HTTP redirects, -r or –range to download only a byte range useful for resuming transfers, and -C – to continue an interrupted download.

Authentication

Curl supports several authentication methods. For HTTP basic credentials use -u username:password or -u username and it will prompt for the password interactively. For bearer tokens, common in modern APIs, add a header with -H ‘Authorization: Bearer your_token_here’. For FTP connections you can specify -u for username and password, and for SFTP you usually combine with SSH keys using the –key and –cert options if required.

Debugging and Verbose

When a request does not behave as expected, the -v option is indispensable. It shows TLS negotiation information, sent and received headers, and the response body. To save this log to a file you can combine it with -o output.txt or redirect standard output to a file. Additionally, curl offers –trace-ascii file.txt that dumps the entire exchange in a readable format for deeper analysis.

Conclusion

The curl command is a Swiss army knife for anyone working with networks on Linux. Its ability to handle multiple protocols, its wealth of options, and its ease of use in scripts make it an indispensable tool. Mastering curl not only speeds up download and data transfer tasks, but also opens the door to sophisticated automation of web services and infrastructure monitoring.

This post is also available in ESPAÑOL.

Leave a Reply

Your email address will not be published. Required fields are marked *

Esta obra está bajo una Licencia Creative Commons Atribución 4.0 Internacional para Francesc Roig francesc@vivaldi.net .