Subscribe
May
3
I know that most people are probably hooked on using wget to download whatever it is they need to retrieve on their systems. I use it in my scripts, crontab and anything else that need to be fetched locally to make life easier. But wget has it’s limitations when it comes to downloading large files in a short period of time. The following is a list of alternatives that’ll blow wget away. All of the applications work on Unix/Linux, MacOS, Cygwin and probably everything else if you can get them compiled.
- Recently, I came across a little program called Axel. Axel tries to accelerate downloads by using multiple connections for one download. It opens more than one HTTP/FTP connection per download and each connection transfers its own, separate, part of the file. This comes handy when some sites limit the speed of each connection so by opening more than one connection at a time multiplies the allowable bandwidth. Once all parts of the file is downloaded, it is seamlessly put together into original. This makes it better than wget and a more qualified tool for downloading large files quicker.
- The second app that rocks wget is Aget. Aget is a multi-threaded download accelerator. It supports HTTP downloads and can be run from the console. From the Aget website: Tests show that Aget is successfull in realizing its objectives. A file of size 36.347.010 bytes was downloaded in 14 minutes 28 secs via wget; whereas it was downloaded in 3 minutes and 15 seconds via aget. Amazing indeed.
- The third super getter is Prozilla. Prozilla is a download accelerator for Linux. It makes multiple server connections to download a file, and also supports FTP searching for faster mirrors. Supports FTP & HTTP and the file will be downloaded as fast as possible as your bandwidth allows if not otherwise specified.
- The fourth rock star is Manda. Manda is a threaded download accelerator. It downloads files via HTTP by splitting the files into parts, speeding the download. I don’t have too much info on this as I didn’t get to test it but it’s in the same rank as axel and aget.
- The fifth and last but definetly not least is GetFast. GetFast is a file download accelerator. It is multi-threaded and allows the downloading of Web pages and their sub-links. GetFast is an animal like the rest. It fetches anything and everything in lightening speed.
So what do we learn from this article? Don’t be afraid to try out new things! Wget is an amazing tool with lots of options and is perfect for many things but if you’re looking for speed and getting large things done in small time, then give these apps a try. And the best part is, unlike wget, these apps resume from where it ended in case download process gets interrupted.
I am probably missing other download accelerators that are on the same rank as the ones above so please put it up in comment area and I will update the article to include all.
Comments
10 Comments so far
From the wget man page:
-c
–continue
Continue getting a partially-downloaded file. This is useful when you want to finish up a download started by a previous
instance of Wget, or by another program. For instance:
wget -c ftp://sunsite.doc.ic.ac.uk/ls-lR.Z
Also, doesn’t using multiple connection downloaders make one a bad Internet citizen with the overloading of servers?
Just to clarify my previous post: the wget man-page-quote ended with the line:
‘wget -c ftp://sunsite.doc.ic.ac.uk/ls-lR.Z’
and the rest of the post is my comment.
And what about aria2? From aria2 sourceforge page (http://aria2.sourceforge.net/):
aria2 is a download utility with resuming and segmented downloading.
Supported protocols are HTTP/HTTPS/FTP/BitTorrent. It also supports Metalink version 3.0. As of 0.10.0 aria2 supports chunk checksum in Metalink: it validates chunk checksums while downloading a file like BitTorrent. This gives safer and more reliable download experience.
Becareful with Axel as too many connections could cause the host to block you temporarily as it may think your DDosing or trying to bring it down.
@Heikki
Thanks for pointing this out. My mistake in not doing enough research on wget on this particular feature.
@Rafa, I have to give aria2 a try. Is it multi-threaded like the ones I’ve posted?
Curl?
http://curl.haxx.se/
I have tried both axel and prozilla. Both of them are good but axel gives very good speeds, but only downside of axel is that there in no active development going on and axel has some other minor issues as well.
So, Right now I am using KGet which I think uses wget as the backend. (Not sure about this.)
degf
axel tended to have high cpu usage when I tested it out, this impacted the speeds that the file was transferring at when compared to aget. aget was using a very small footprint and was giving me about 250kbps versus axel at 175kbps.
Definitely useful when you have a remote server of your own where you have a repository of files you want quick access to locally over a slower connection. Most ISPs - unless u r in the free (as in little restriction) Internet worlds in Japan/Sth Korea - tend to shape traffic per connection to some degree, especially and peak usage times.
aria2 sounds like it has some potential - will look at at that too - thnx for the heads up.
This is are cool, but Im looking for a windows alternative… multithread and with progress bar indicators. Thanks