In this article, you will explore the power of the wget command, discover the key features of wget, learn how to easily install it on major Linux distributions, and gain hands-on experience with practical examples.
|Root or Sudo Privileges||Yes|
|Host System and Architecture||Ubuntu 22.10 (x64)|
|OS Compatibility||Ubuntu, Manjaro, Fedora, etc.|
What is the Wget Command in Linux?
Wget (also known as network downloader) is a widely used command-line utility that is capable of downloading files from the web using various protocols, such as HTTP, HTTPS, FTP, and FTPS.
It’s a non-interactive network downloader that’s capable of downloading files without authentication or downloading files in the background without any interruptions.
Consider it a reliable and versatile tool that can be used in shell scripts, automation, and command-line environments to efficiently download files and automate the retrieval process.
The workings of wget and curl might seem identical, as they serve similar purposes. While curl is known for its ability to perform more advanced operations like sending requests and interacting with web services, wget is primarily focused on file downloading functionality.
Wget is a feature-rich command-line utility that offers a wide range of capabilities. Here’s a glimpse of some of the powerful features wget brings to the table:
- Recursively download entire directories.
- Download files through proxies.
- Follow redirects (hop between pages).
- Resume interrupted downloads.
- Limit the download rate.
- Specify the output file names.
- Put the download process in the background.
- Support for IPv4 and IPv6 downloads.
- Adjust timeouts and handle authentication.
The great thing about this tool is that it works seamlessly on Windows, MacOS, and Linux/UNIX. Once you learn how to use it on one platform, you can easily operate it on others as well.
How to Install Wget on Linux
Wget is a widely used and popular utility that comes pre-installed in major Linux distributions. However, if, for any reason, your distribution does not include it, you can easily install it using the package manager specific to your Linux distribution.
Here are the commands for some popular distributions:
$ sudo apt install wget -y #Debian, Ubuntu, or Pop!_OS $ sudo yum install wget -y #Red Hat, Fedora, or AlmaLinux $ sudo pacman -Sy wget #Arch, Manjaro, or EndeavourOS $ sudo zypper install wget -y #OpenSUSE $ sudo emerge -a net-misc/wget #Gentoo Linux
After successfully installing wget, you can verify its accessibility by opening the terminal and running the following command:
$ wget --version
How to Use the Wget Command on Linux
The wget command requires two arguments: one is the option, and the other is the URL.
$ wget [OPTION] [URL]
The following are some examples of how you can use wget for various tasks:
1. Download a File from a URL
You can specify a URL containing a file that will be downloaded in the current directory. While downloading, it will display the download progress, server IP address, size, date, and time.
$ wget https://ftp.gnu.org/gnu/wget/wget2-latest.tar.gz
2. Download Multiple Files from Different URLs
Specify each URL as an argument to this command, using space as a separator.
$ wget https://ftp.gnu.org/gnu/wget/wget2-latest.tar.gz https://ftp.gnu.org/gnu/wget/wget2-latest.tar.lz
Alternatively, you can save the URLs in a text file and specify the file with the “
$ cat urls.txt $ wget -i urls.txt
3. Download Multiple Files with HTTP and FTP Protocols
As I’ve mentioned earlier, this command supports a variety of protocols. So, you can specify each URL with a different protocol to download files from them.
$ wget https://ftp.gnu.org/gnu/wget/wget2-latest.tar.gz ftp://ftp.gnu.org/gnu/wget/wget2-latest.tar.gz.sig
4. Download a File and Save it with a Different Name
Using the “
-O” (capital “O“) flag, you can specify a different name for the file you want to download from the wget command.
$ wget -O wget.tar.gz https://ftp.gnu.org/gnu/wget/wget2-latest.tar.gz
5. Download a File and Specify the Output Directory
The default nature of the wget command (without any flags) is to pull the file from the server and save it in the current directory.
However, you can alter this process and download and save your file to your desired location using the “
-P” (capital “P”) flag.
$ wget -P ~/Downloads/ https://ftp.gnu.org/gnu/wget/wget2-latest.tar.gz
6. Download a File Using a Specific User Agent
You can download your file using a specific user agent by using the “
--user-agent” flag and providing the desired user agent string.
$ wget --user-agent="Mozilla/5.0 (X11; Linux i686; rv:109.0) Gecko/20100101 Firefox/115.0" https://ftp.gnu.org/gnu/wget/wget2-latest.tar.gz
7. Download a Large File with Resumable Capability
Whenever you download a large file, like an ISO image of any Linux distribution, it might happen for any specific reason that you don’t want to download the whole file in one go.
On such occasions, you can use the “
-c” (small “c”) flag with your command to download the same file where it was left without starting from scratch.
-c” flag, a fresh file will be downloaded with the “
.1” extension at the end.
$ wget -c https://cdimage.debian.org/debian-cd/current/amd64/iso-dvd/debian-12.0.0-amd64-DVD-1.iso
8. Download a File and Limit the Download Rate
Downloading a large file might consume a lot of time (depending on your internet speed) and interfere with other work as most bandwidth will be used in the downloading of that file.
However, you can use the “
--limit-rate=500k” flag to restrict the download speed limit to “
500k“, and the “
wget.log” file will be created.
$ wget -c --limit-rate=500k https://cdimage.debian.org/debian-cd/current/amd64/iso-dvd/debian-12.0.0-amd64-DVD-1.iso
9. Download the File in the Background
When downloading a large file, the process might take time. In that case, you can use the “
-b” (small “b”) flag to download it in the background, and information related to the background download process will be written on the “
$ wget -c --limit-rate=500k -b https://cdimage.debian.org/debian-cd/current/amd64/iso-dvd/debian-12.0.0-amd64-DVD-1.iso
Once the download is running in the background, you can continue to use your terminal for other commands. To stop the download process, you will have to find the process ID (or PID) of the background process using the ps command.
$ ps aux | grep wget
Identify the PID of the specific wget process you want to stop, and then use the kill command:
$ kill 7681
10. Mirror a Website (Download All Files and Directories Recursively)
To mirror a website using the wget command, it can be easily done by using the following command:
$ wget --mirror --convert-links --adjust-extension --page-requisites --no-parent https://example.com
The following is a breakdown of all the flags used in the above command:
--mirror“: Enables mirroring mode for recursive downloading.
--convert-links“: Update the links to work locally, allowing for offline browsing.
--adjust-extension“: Assign suitable extensions to downloaded files.
--no-parent“: Does not follow the links outside the directory you pass in.
11. Download the Password-Protected File via HTTP and FTP
To download a file from a password protected HTTP server using the wget command, you can use the “
--http-user=USER” and “
$ wget --http-user=USER --http-password=PASSWORD https://example.com/file.zip
To download a file from a password protected FTP server using the wget command, you can use the “
--ftp-user=USER” and “
$ wget --ftp-user=USER --ftp-password=PASSWORD ftp://example.com/file.zip
12. Download File Using Cookie
To download a file using the cookie with the wget command, you can use the “
--load-cookies” flag to specify a file containing the cookie data.
$ wget --load-cookies cookies.txt https://example.com/protected-file.zip
13. Ignore the SSL Certificate Check
To ignore SSL certificate checks in wget, you can use the “
--no-check-certificate” flag. This option tells wget not to validate SSL certificates, allowing you to download files from SSL-protected sites without encountering certificate errors.
$ wget --no-check-certificate https://example.com/file.zip
14. Check the Wget Help and Manual Page
To learn more about the wget command, you can check the help section to get a summary view of all the available flags or read the comprehensive manual page by using the following commands:
$ wget --help $ man wget
In this article, I tried to cover all the important usages of wget commands with practical examples. Although, if you have stuck somewhere or have any questions or queries related to this article, feel free to ask them in the comment section.
Till then, peace!