• Uncategorized

About linux : Is-it-possible-to-adapt-wgets-progress-bar-to-several-files

Question Detail

I’m downloading all the “.htm” files of some directories by doing more or less:

wget http://some/url/ -r --accept="*.htm" -nv --show-progress

In which i turned off wget’s printing but kept the progress bar which is useful in my case (-nv --show-progress)

This works great but outputs a progress bar for every downloaded file.
Is it a possible to have a single progress bar that would take into account the sum of all the sizes of the files?

I looked at the the progress=TYPE option but this only seems to setup the style of the progress bar and not the total amount of downloaded data.

Question Answer

No, currently there is no way in Wget to have a single aggregate bar.

However, you can try the alpha version of Wget 2.0. It’s not exactly what you’re looking for, but comes very close. It has been packages as Wget2 in Debian and is available on Arch Linux’s AUR. I’m not sure about other distros.

wget2 supports parallel downloads and HTTP/2 by default and a line under the progress bars showing some aggregate stats. For example:

$ wget2 --progress=bar "example.com/?"{0,1,2,3,4,5,6,7,8}
index.html?8         100% [========================================================================================================================>]     606    32,88KB/s
index.html?5         100% [========================================================================================================================>]     606    18,49KB/s
index.html?6         100% [========================================================================================================================>]     606    31,15KB/s
index.html?7         100% [========================================================================================================================>]     606    32,88KB/s
index.html?4         100% [========================================================================================================================>]     606    34,81KB/s
                          [Files: 9  Bytes: 5,33K [11,78KB/s] Redirects: 0  Todo: 0  Errors: 0                                       ]

You see 5 progress bars because 5 threads were used to download the 9 files in parallel. The last bar indicates aggregate stats.

You can easily build Wget2 from git or using the v1.99 tarball available here: https://alpha.gnu.org/gnu/wget/wget2-1.99.0.tar.gz

DISCLAIMER: I maintain both GNU Wget and Wget2.

You may also like...

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.