Networking and remote

`wget`

Download files over HTTP, HTTPS and FTP, non-interactively, with retries, resuming and recursive mirroring.

wget [OPTION...] URL...

13 of 13 examples run in Debian 13.7 on 2026-10-31 · wget 1.25.0-2 · manual: wget(1)

wget URL downloads to a file named after the URL, in the current directory, and narrates what it is doing: resolving, connecting, the response status, the size and a progress bar. It was built to run unattended: it retries on network errors, can resume a partial file with -c, and can follow links recursively to mirror part of a site.

Compared with curl, wget saves to a file by default (curl prints to standard output), follows redirects by default, and exits non-zero on HTTP errors by default. curl is the better tool for talking to APIs (methods, headers, JSON bodies, -w output); wget is the quicker one for "fetch these files" and for mirroring.

The examples use a test server at 127.0.0.1:8080 inside the sandbox; the same commands work with any host name. (On a machine where localhost resolves to IPv6 ::1 first and the server only listens on IPv4, wget prints a failed: Connection refused line before falling back to 127.0.0.1: noisy but harmless.)

Examples

Download a file

$ wget http://127.0.0.1:8080/files/report.csv

explain

output

--2026-10-31 20:22:06--  http://127.0.0.1:8080/files/report.csv
Connecting to 127.0.0.1:8080... connected.
HTTP request sent, awaiting response... 200 OK
Length: 405 [text/csv]
Saving to: ‘report.csv’

     0K                                                       100% 68.0M=0s

2026-10-31 20:22:06 (68.0 MB/s) - ‘report.csv’ saved [405/405]

The default report: the request, 200 OK, the length and type, a progress line and a summary with the saved name and byte count. The file is named after the last part of the URL.

One line per file with -nv

$ wget -nv http://127.0.0.1:8080/index.html http://127.0.0.1:8080/files/report.csv

explain

output

2026-10-31 20:22:06 URL:http://127.0.0.1:8080/index.html [152/152] -> "index.html" [1]
2026-10-31 20:22:06 URL:http://127.0.0.1:8080/files/report.csv [405/405] -> "report.csv" [1]
FINISHED --2026-10-31 20:22:06--
Total wall clock time: 0.04s
Downloaded: 2 files, 557 in 0.04s (13.5 KB/s)

-nv (no verbose) reduces each download to one line with the URL, size and file name; with several URLs it adds a total. -q would print nothing at all.

Choose the file name with -O

$ wget -q -O servers.json http://127.0.0.1:8080/api/servers.json
> head -c 120 servers.json; echo

output

[
  {"name": "web01", "ip": "10.0.0.11", "role": "web", "cpus": 4, "tags": ["prod", "eu"], "up": true},
  {"name": "web0

-O FILE saves under your chosen name. Note it is a capital O; unlike curl's -O, it takes an argument.

Print to standard output for a pipeline

$ wget -qO- http://127.0.0.1:8080/api/status.json | jq -r .version

explain

output

2.4.1

-O - writes the body to standard output and -q keeps the log messages away, so the JSON goes straight to jq. -qO- is the usual compact spelling.

Save into a directory with -P

$ wget -q -P downloads http://127.0.0.1:8080/files/report.csv
> ls downloads

output

report.csv

-P DIR (directory prefix) saves the file under DIR, creating it if needed.

Download every URL listed in a file

urls.txt

http://127.0.0.1:8080/api/status.json
http://127.0.0.1:8080/index.html

$ wget -nv -i urls.txt

explain

output

2026-10-31 20:22:07 URL:http://127.0.0.1:8080/api/status.json [56/56] -> "status.json" [1]
2026-10-31 20:22:07 URL:http://127.0.0.1:8080/index.html [152/152] -> "index.html" [1]
FINISHED --2026-10-31 20:22:07--
Total wall clock time: 0.04s
Downloaded: 2 files, 208 in 0.04s (4.97 KB/s)

-i FILE reads one URL per line. wget fetches them in order and prints a summary at the end.

Skip files that have not changed with -N

Setup run before this example

wget -q -N http://127.0.0.1:8080/files/report.csv

$ wget -N http://127.0.0.1:8080/files/report.csv

explain

output

--2026-10-31 20:22:07--  http://127.0.0.1:8080/files/report.csv
Connecting to 127.0.0.1:8080... connected.
HTTP request sent, awaiting response... 304 Not Modified
File ‘report.csv’ not modified on server. Omitting download.

report.csv was already downloaded with -N, which gives the local file the server's modification time. This time wget asks the server whether it changed since, receives 304 Not Modified and keeps the existing file. Good for refreshing a set of downloads in a cron job.

Check a URL without downloading it

$ wget -S --spider http://127.0.0.1:8080/old-page

explain

output

Spider mode enabled. Check if remote file exists.
--2026-10-31 20:22:07--  http://127.0.0.1:8080/old-page
Connecting to 127.0.0.1:8080... connected.
HTTP request sent, awaiting response... 
  HTTP/1.1 301 Moved Permanently
  Server: demo-httpd/1.0
  Date: Sat, 31 Oct 2026 20:22:07 GMT
  Location: /index.html
  Content-Length: 0
Location: /index.html [following]
Spider mode enabled. Check if remote file exists.
--2026-10-31 20:22:07--  http://127.0.0.1:8080/index.html
Connecting to 127.0.0.1:8080... connected.
HTTP request sent, awaiting response... 
  HTTP/1.1 200 OK
  Server: demo-httpd/1.0
  Date: Sat, 31 Oct 2026 20:22:07 GMT
  Content-type: text/html
  Content-Length: 152
  Last-Modified: Mon, 26 Oct 2026 13:48:10 GMT
Length: 152 [text/html]
Remote file exists and could contain further links,
but recursion is disabled -- not retrieving.

--spider only checks that the file exists, and -S prints the server's response headers. wget follows the 301 to /index.html on its own and reports that the target exists.

Mirror a directory recursively

$ wget -r -np -nH -nv http://127.0.0.1:8080/api/
> find api -type f

output

2026-10-31 20:22:07 URL:http://127.0.0.1:8080/api/ [291/291] -> "api/index.html" [1]
http://127.0.0.1:8080/robots.txt:
2026-10-31 20:22:07 ERROR 404: File not found.
2026-10-31 20:22:07 URL:http://127.0.0.1:8080/api/servers.json [503/503] -> "api/servers.json" [1]
2026-10-31 20:22:07 URL:http://127.0.0.1:8080/api/status.json [56/56] -> "api/status.json" [1]
FINISHED --2026-10-31 20:22:07--
Total wall clock time: 0.04s
Downloaded: 3 files, 850 in 0.04s (20.5 KB/s)
api/servers.json
api/index.html
api/status.json

-r follows links, -np (no parent) stays below /api/, and -nH drops the host:port directory wget would otherwise create. The server's directory listing becomes api/index.html, and both JSON files it links to are fetched. wget also asks for robots.txt first and respects it; the 404 for it is harmless.

Send a POST request

$ wget -qO- --post-data 'name=web03&role=web' http://127.0.0.1:8080/api/servers

explain

output

{
  "method": "POST",
  "path": "/api/servers",
  "headers": {
    "User-Agent": "Wget/1.25.0",
    "Accept": "*/*",
    "Content-Type": "application/x-www-form-urlencoded"
  },
  "body": "name=web03&role=web"
}

--post-data sends a form-encoded body. The test server echoes what it got. For anything beyond simple forms, curl is more convenient.

Authenticate with a user name and password

$ wget -q --user=alice --password=s3cret http://127.0.0.1:8080/private/report.txt
> cat report.txt

output

Quarterly report: confidential

wget sends Basic credentials after the server's 401 challenge. --ask-password prompts instead, which keeps the password out of your history and the process list.

An HTTP error fails the command

$ wget http://127.0.0.1:8080/status/404

explain

outputexit 8

--2026-10-31 20:22:07--  http://127.0.0.1:8080/status/404
Connecting to 127.0.0.1:8080... connected.
HTTP request sent, awaiting response... 404 Not Found
2026-10-31 20:22:07 ERROR 404: Not Found.

wget reports ERROR 404, saves nothing and exits with status 8, its code for "the server issued an error response". Scripts can rely on it without extra options.

When nothing is listening

$ wget -nv --tries=3 http://127.0.0.1:9999/file

explain

outputexit 4

failed: Connection refused.

A refused connection is a network failure: exit status 4. wget treats "connection refused" as final and does not use up its --tries; add --retry-connrefused if the server may still be starting.

Common pitfalls

wget -O file URL1 URL2 concatenates both downloads into one file. Use -P or separate commands for separate files.
Running wget URL twice gives you report.csv and report.csv.1. Use -N (timestamping), -nc (no clobber) or -O to control it.
--password=... on the command line is visible in ps and in your history. Prefer --ask-password or a ~/.netrc file with mode 600.
Recursive mode without -np can climb to the site root and download far more than you meant. Add -np, and -l DEPTH to limit depth.
-c (continue) only resumes if the server supports byte ranges; otherwise the download starts again from the beginning.

GNU vs BSD and macOS

The examples above ran the GNU/Linux version. These are the differences that matter on macOS and the BSDs (written from their documentation, not run here).

macOS does not ship wget; install it from Homebrew or MacPorts, or use the built-in curl -O. FreeBSD has fetch in the base system.

Options worth knowing

A selection, described in our own words. For the full list, see wget(1) or wget --help.

Option	What it does
`-O, --output-document FILE`	Write the download to FILE; - means standard output.
`-P, --directory-prefix DIR`	Save files under DIR.
`-q, --quiet`	Print nothing.
`-nv, --no-verbose`	Print one line per file instead of the full report.
`-c, --continue`	Resume a partially downloaded file.
`-N, --timestamping`	Download only if the remote file is newer than the local one.
`-nc, --no-clobber`	Do not overwrite or rename around an existing file.
`-i, --input-file FILE`	Read URLs from FILE.
`-t, --tries N`	Try each download up to N times (default 20).
`--retry-connrefused`	Retry even when the connection is refused.
`-T, --timeout SECONDS`	Set DNS, connect and read timeouts.
`--limit-rate RATE`	Limit download speed, for example 500k.
`-S, --server-response`	Print the server's response headers.
`--spider`	Check that pages exist without downloading them.
`-r, --recursive`	Follow links and download recursively.
`-l, --level DEPTH`	Maximum recursion depth (default 5).
`-np, --no-parent`	Never ascend above the starting directory when recursing.
`-nH, --no-host-directories`	Do not create a directory named after the host.
`-m, --mirror`	Mirror: recursive, infinite depth, with timestamping.
`-k, --convert-links`	Rewrite links in downloaded pages to work locally.
`-p, --page-requisites`	Also download images, CSS and scripts a page needs.
`-A, --accept LIST`	Only keep files matching these suffixes or patterns.
`-R, --reject LIST`	Skip files matching these suffixes or patterns.
`--user USER`	User name for HTTP or FTP authentication.
`--password PASS`	Password for authentication (visible in ps).
`--ask-password`	Prompt for the password.
`--header HEADER`	Add a request header.
`--post-data DATA`	Send DATA in a POST request.
`-U, --user-agent AGENT`	Set the User-Agent header.
`--no-check-certificate`	Do not verify the server's TLS certificate (testing only).

Examples

Common pitfalls

GNU vs BSD and macOS

Options worth knowing

Related