Tip: Download Accelerator for Linux

There are different ways of downloading files from remote locations. While all web browsers support downloading files via http or ftp, I have found that in the case of Firefox (at least), it doesn't support resuming downloads. While this is not a problem for files of small size, when it Linux distributions, this lack of support for resuming downloads from where you had left off earlier would mean you have to start downloading the file again from the beginning.

The most common and fail safe method of downloading huge files in Linux is to use the wget tool. of the file at a later stage in the event the downloading fails due to connection time out.

I usually use the following wget command to download Linux distributions.

$ wget -c full_path_to_the_linux_iso

But here is a nice tip to speed up your download of files by a significant factor. This tip is and above. Usually different mirrors have different speeds. A Linux distribution mirror in say Japan may have a 100MBps connection but another mirror in a different location will be connected only to a 10 MBps pipe. More over these mirrors throttle the bandwidth made available for each individual connection thus providing a upper limit of download speeds.

What you can do is split the file (in our case the ISO image) you are downloading into a number of pieces and download each piece from a different mirror simultaneously. At the end of the download, you can combine all the pieces together to get your file in one piece. This is known as download acceleration where the software connects to more than one location simultaneously and splits the downloads among the locations. This feature is commonly found in many download managers for Windows but is hard to find in those available for Linux.

You can get the same feature in Linux too by using the program curl to download your files.

$ curl --range 200000000-399999999 -o ubuntu-iso.part2 $url2 &
$ curl --range 400000000-599999999 -o ubuntu-iso.part3 $url3 &
$ curl --range 600000000- -o ubuntu-iso.part4 $url4 &

Where url1, url2, url3 and url4 are as follows ...

url1=http://ubuntu.intergenia.de/releases/feisty/ubuntu-7.04-desktop-i386.iso
url2=http://ubuntu.virginmedia.com/releases/7.04/ubuntu-7.04-desktop-i386.iso
url3=http://es.releases.ubuntu.com/7.04/ubuntu-7.04-desktop-i386.iso
url4=http://ftp-mirror.internap.com/pub/ubuntu-releases/7.04/ubuntu-7.04-desktop-i386.iso

This creates four background download processes, each transferring a different part of the ISO specifies a subrange of bytes to extract from the ISO file. The -o option provides the name of the file to which the data is to be saved. So once all the four curl processes finish their download, you will have four files namely ubuntu-iso.part1, ubuntu-iso.part2, ubuntu-iso.part3 and ubuntu-iso.part4 in your current directory.

To get the original Ubuntu ISO file, I just combine the files using the cat command as follows :

$ cat ubuntu-iso.part? > ubuntu-7.04-desktop-i386.iso

As I said earlier, if you have a high speed Internet connection then this method will considerably shorten the time taken to download huge files.

As an aside, I have entered all the commands in a file as follows :

#FILE NAME : da.sh (Download accelerator)
#!/bin/sh

url1=http://ubuntu.intergenia.de/releases/feisty/ubuntu-7.04-desktop-i386.iso
url2=http://ubuntu.virginmedia.com/releases/7.04/ubuntu-7.04-desktop-i386.iso
url3=http://es.releases.ubuntu.com/7.04/ubuntu-7.04-desktop-i386.iso
url4=http://ftp-mirror.internap.com/pub/ubuntu-releases/7.04/ubuntu-7.04-desktop-i386.iso

curl --range 0-199999999 -o ubuntu-iso.part1 $url1 &
curl --range 200000000-399999999 -o ubuntu-iso.part2 $url2 &
curl --range 400000000-599999999 -o ubuntu-iso.part3 $url3 &
curl --range 600000000- -o ubuntu-iso.part4 $url4 &

... set the executable bit of the file and then I run it.

$ chmod u+x da.sh
$ ./da.sh