Installing and Using Wget on Linux Like a Pro

In this article, you will explore the power of the wget command, discover the key features of wget, learn how to easily install it on major Linux distributions, and gain hands-on experience with practical examples.

Tutorial Details

What is the Wget Command in Linux?

Wget (also known as network downloader) is a widely used command-line utility that is capable of downloading files from the web using various protocols, such as HTTP, HTTPS, FTP, and FTPS.

It’s a non-interactive network downloader that’s capable of downloading files without authentication or downloading files in the background without any interruptions.

Consider it a reliable and versatile tool that can be used in shell scripts, automation, and command-line environments to efficiently download files and automate the retrieval process.

The workings of wget and curl might seem identical, as they serve similar purposes. While curl is known for its ability to perform more advanced operations like sending requests and interacting with web services, wget is primarily focused on file downloading functionality.

Key Features

Wget is a feature-rich command-line utility that offers a wide range of capabilities. Here’s a glimpse of some of the powerful features wget brings to the table:

• Recursively download entire directories.
• Download files through proxies.
• Follow redirects (hop between pages).
• Resume interrupted downloads.
• Limit the download rate.
• Specify the output file names.
• Put the download process in the background.
• Support for IPv4 and IPv6 downloads.
• Adjust timeouts and handle authentication.

The great thing about this tool is that it works seamlessly on Windows, MacOS, and Linux/UNIX. Once you learn how to use it on one platform, you can easily operate it on others as well.

How to Install Wget on Linux

Wget is a widely used and popular utility that comes pre-installed in major Linux distributions. However, if, for any reason, your distribution does not include it, you can easily install it using the package manager specific to your Linux distribution.

Here are the commands for some popular distributions:

$ sudo apt install wget -y                                                                          #Debian, Ubuntu, or Pop!_OS
$ sudo yum install wget -y                                                                        #Red Hat, Fedora, or AlmaLinux
$ sudo pacman -Sy wget                                                                           #Arch, Manjaro, or EndeavourOS 
$ sudo zypper install wget -y                                                                    #OpenSUSE
$ sudo emerge -a net-misc/wget                                                            #Gentoo Linux

After successfully installing wget, you can verify its accessibility by opening the terminal and running the following command:

$ wget --version

Output:

How to Use the Wget Command on Linux

The wget command requires two arguments: one is the option, and the other is the URL.

$ wget [OPTION] [URL]

The following are some examples of how you can use wget for various tasks:

1. Download a File from a URL

You can specify a URL containing a file that will be downloaded in the current directory. While downloading, it will display the download progress, server IP address, size, date, and time.

$ wget https://ftp.gnu.org/gnu/wget/wget2-latest.tar.gz

Output:

2. Download Multiple Files from Different URLs

Specify each URL as an argument to this command, using space as a separator.

$ wget https://ftp.gnu.org/gnu/wget/wget2-latest.tar.gz https://ftp.gnu.org/gnu/wget/wget2-latest.tar.lz

Output:

Alternatively, you can save the URLs in a text file and specify the file with the “-i” flag.

$ cat urls.txt
$ wget -i urls.txt

Output:

3. Download Multiple Files with HTTP and FTP Protocols

As I’ve mentioned earlier, this command supports a variety of protocols. So, you can specify each URL with a different protocol to download files from them.

$ wget https://ftp.gnu.org/gnu/wget/wget2-latest.tar.gz ftp://ftp.gnu.org/gnu/wget/wget2-latest.tar.gz.sig

Output:

4. Download a File and Save it with a Different Name

Using the “-O” (capital “O“) flag, you can specify a different name for the file you want to download from the wget command.

$ wget -O wget.tar.gz https://ftp.gnu.org/gnu/wget/wget2-latest.tar.gz

Output:

5. Download a File and Specify the Output Directory

The default nature of the wget command (without any flags) is to pull the file from the server and save it in the current directory.

However, you can alter this process and download and save your file to your desired location using the “-P” (capital “P”) flag.

$ wget -P ~/Downloads/ https://ftp.gnu.org/gnu/wget/wget2-latest.tar.gz

Output:

6. Download a File Using a Specific User Agent

You can download your file using a specific user agent by using the “--user-agent” flag and providing the desired user agent string.

$ wget --user-agent="Mozilla/5.0 (X11; Linux i686; rv:109.0) Gecko/20100101 Firefox/115.0" https://ftp.gnu.org/gnu/wget/wget2-latest.tar.gz

Output:

7. Download a Large File with Resumable Capability

Whenever you download a large file, like an ISO image of any Linux distribution, it might happen for any specific reason that you don’t want to download the whole file in one go.

On such occasions, you can use the “-c” (small “c”) flag with your command to download the same file where it was left without starting from scratch.

$ wget -c https://cdimage.debian.org/debian-cd/current/amd64/iso-dvd/debian-12.0.0-amd64-DVD-1.iso

Output:

8. Download a File and Limit the Download Rate

Downloading a large file might consume a lot of time (depending on your internet speed) and interfere with other work as most bandwidth will be used in the downloading of that file.

However, you can use the “--limit-rate=500k” flag to restrict the download speed limit to “500k“, and the “wget.log” file will be created.

$ wget -c --limit-rate=500k https://cdimage.debian.org/debian-cd/current/amd64/iso-dvd/debian-12.0.0-amd64-DVD-1.iso

Output:

9. Download the File in the Background

When downloading a large file, the process might take time. In that case, you can use the “-b” (small “b”) flag to download it in the background, and information related to the background download process will be written on the “wget.log” file.

$ wget -c --limit-rate=500k -b https://cdimage.debian.org/debian-cd/current/amd64/iso-dvd/debian-12.0.0-amd64-DVD-1.iso

Output:

Once the download is running in the background, you can continue to use your terminal for other commands. To stop the download process, you will have to find the process ID (or PID) of the background process using the ps command.

$ ps aux | grep wget

Output:

Identify the PID of the specific wget process you want to stop, and then use the kill command:

$ kill 7681

Output:

10. Mirror a Website (Download All Files and Directories Recursively)

To mirror a website using the wget command, it can be easily done by using the following command:

$ wget --mirror --convert-links --adjust-extension --page-requisites --no-parent https://example.com

The following is a breakdown of all the flags used in the above command:

• “--mirror“: Enables mirroring mode for recursive downloading.
• “--convert-links“: Update the links to work locally, allowing for offline browsing.
• “--adjust-extension“: Assign suitable extensions to downloaded files.
• “--page-requisites“: Retrieves all the assets to display the page correctly, such as images, CSS, and JavaScript.
• “--no-parent“: Does not follow the links outside the directory you pass in.

11. Download the Password-Protected File via HTTP and FTP

To download a file from a password protected HTTP server using the wget command, you can use the “--http-user=USER” and “--http-password=PASSWORD” flags.

$ wget --http-user=USER --http-password=PASSWORD https://example.com/file.zip

To download a file from a password protected FTP server using the wget command, you can use the “--ftp-user=USER” and “--ftp-password=PASSWORD” flags.

$ wget --ftp-user=USER --ftp-password=PASSWORD ftp://example.com/file.zip

12. Download File Using Cookie

To download a file using the cookie with the wget command, you can use the “--load-cookies” flag to specify a file containing the cookie data.

$ wget --load-cookies cookies.txt https://example.com/protected-file.zip

13. Ignore the SSL Certificate Check

To ignore SSL certificate checks in wget, you can use the “--no-check-certificate” flag. This option tells wget not to validate SSL certificates, allowing you to download files from SSL-protected sites without encountering certificate errors.

$ wget --no-check-certificate https://example.com/file.zip

14. Check the Wget Help and Manual Page

To learn more about the wget command, you can check the help section to get a summary view of all the available flags or read the comprehensive manual page by using the following commands:

$ wget --help
$ man wget

Wrap Up

In this article, I tried to cover all the important usages of wget commands with practical examples. Although, if you have stuck somewhere or have any questions or queries related to this article, feel free to ask them in the comment section.

Till then, peace!