Helping people with computers... one answer at a time.

Download accelerators use a number of techniques to make downloads appear faster and more resilient. I'll cover the most common ways accelerators work.

I recently downloaded two sizable content files, each 1.1GB. My FIOS is 20/5. The first file took X minutes to download. The second download link (from the same company) to me to a "content downloader" and the file of the same size took less than half the time. I mean, it was quite noticeably faster. I wonder how their "downloader" program can cut the download time of a huge file in half because my internet connection is still 20/5.

Download accelerators certainly don't make your internet connection any faster, but they do sometimes seem like they do.

A normal file download is essentially a file copy operation. A file on some remote server is copied to your computer.

A download accelerator does that too, but it can make use of a few tricks to simply be more efficient at it; sometimes, much more efficient.

Download Accelerators

There are many download accelerators and I'm not going to speak to any one specific instance.

Rather I'm going to discuss the techniques that they may or may not use to improve your download speeds.

Some accelerators may use some, others may use all of the techniques here. Heck, there are probably accelerators that are using techniques that haven't even thought of.

But they all do share one thing in common.

They do not make your internet connection any faster.

Parallelism

Parallelism is just a fancy word meaning doing two (or more) things at once.

To understand why that might be preferable to doing only one thing at a time, we need to understand how - at a very grossly over-simplified level - a normal download works.

Normally, when you download a file, it's sent in chunks, one chunk at a time. You can think of it something like this:

Your Computer Download Server
I'd like "wondefullprogram.exe" please
Here's chunk #1
... delay ...
OK, got it!
Here's chunk #2
... delay ...
OK, got it!
Here's chunk #3
... delay ...
OK, got it!

You get the idea. Each chunk is sent in turn; after each chunk, your computer spends a little time making sure that it got it correctly. Couple that with other technical overhead in the communications protocol and files rarely travel at "full speed" in a normal download.

The most common way that download accelerators overcome this is to do two (or more) things in parallel.

Connection #1 Connection #2
Your Computer Download Server Your Computer Download Server
I'd like "wondefullprogram.exe" please
Here's chunk #1 send chunk #2 please
Got chunk #1, send #3 Here's chunk #2
Here's chunk #3 Got #2, send #4
Got #3, send #5 Here's chunk #4
Here's chunk #5 Got #4, send #6
Got #5, send #7 Here's chunk #6

While the two downloads compete with each other for the full speed of your internet connection, the delays that we saw earlier are effectively eliminated; if one of the two download connections delays for any reason, that simply means that the other one will continue at a faster pace.

In effect, this is also what your web browser does when it renders a web page. Separate connections are created for each of the elements on the page - one for the page itself, and then as they are encountered, one for each item, such as an image, a style sheet, a javascript file - and so on to some pre-configured maximum number of connections.

Normally, in a traditional file, download only one connection is used. Download accelerators most commonly speed up downloads by creating more than one. Interestingly, the impact of this technique appears to be greatest the faster your internet connection, where the delays in a normal download become a proportionally larger percentage of the time spent downloading.

Compression

If a file being download is not itself compressed - say it's a text file or word document, or even an executable ".exe" file - the act of downloading it does not necessarily compress it.

Some download accelerators cause the file to be compressed by the server before sending it to your computer and then automatically decompress it when it arrives. Because compression makes the file smaller in most cases, that means that fewer bytes are being downloaded across your internet connection. The net effect is that the download happens faster.

This doesn't work for all file types; files that are already compressed, like .jpg pictures or .zip archives don't benefit from this last-minute compression. Because "compressing" an already-compressed file can actually make it bigger, a good download accelerator will pay attention to the type of file being downloaded and only compress it if there would be a net savings.

Resumability

This isn't so much a true speed up in the downloading technology, but it represents a huge speed up if you've ever been disconnected in the middle of a large download.

Normally, if your computer somehow loses its connection to the download server in the middle of a large download, you have no choice but to start over from the beginning. Imagine being 95% of the way through that 1.1GB download, only to have it fail. Now that 1.1GB download is going to end up taking almost twice as long because you had to download it 1.95 times - once to get it to work, plus that 0.95 that failed (assuming the second attempt is successful).

Many download accelerators not only magically remember where you were when a download is disconnected for some reason, but they'll often automatically re-connect. You might never even know that anything was amiss. Your download simply happens.

What's the Catch?

So if this acceleration is so great, why don't all downloads just do it normally?

Two reasons combine:

  • Most of the techniques used to speed up downloads, such as those which I describe above, require compatible support from the server that hosts the download.

  • There's no standard.

What that means is that you can't just install an accelerator and have it work everywhere - it requires support from the sites from which you are downloading.

It also means that the most common tool to perform downloads - your web browser - can't just add this feature.

† One can argue that the bittorrent protocol is a standard. Technically, it's not a download in the "one server to your machine" sense, but rather, it's a technique that collects the various "chunks" of a file from other computers - all peers - hosting a copy of that same file. But the protocol definitely makes heavy use of parallel connections to maximize throughput.

Article C5072 - February 9, 2012 « »

Share this article with your friends:

Share this article on Facebook Tweet this article Email a link to this article
Leo Leo A. Notenboom has been playing with computers since he was required to take a programming class in 1976. An 18 year career as a programmer at Microsoft soon followed. After "retiring" in 2001, Leo started Ask Leo! in 2003 as a place for answers to common computer and technical questions. More about Leo.

Not what you needed?

9 Comments
duane
February 10, 2012 6:04 AM

This must be how Microsoft updates works. It seems to pause the downloads when you go off line and continue when you come back on line. I would think this "pause" method would save a whole lot of traffic across the Internet. So much that some administrative group would be looking into it.

Gordintoronto
February 10, 2012 2:03 PM

To understand the question, it might help to know what "My FIOS is 20/5" means. Huh?

FIOS is a internet connection technology like DSL or cable, that uses fiber-optic connections for higher speeds. 20/5 means 20 megabits per second download speed, and 5 mbps upload.
Leo
10-Feb-2012
Austin
February 10, 2012 8:21 PM

I've noticed that when I buy an MP3 disk from Amazon or ClassicsOnline they both ask me first to download and use their own download program which in turn downloads a whole CD very fast indeed - much faster than other downloads. Clearly they're using an accelerator that's compatible with their end of things.

Johan
February 10, 2012 9:11 PM

Hi Leo! Thank you for your article.

A couple of questions.

There is something quite peculiar going on when downloading any YouTube file using one of the download add-ons (Download Helper) to my Firefox.

Throughout the download, the Firefox's download window displays the progress: a sliding bar indicator, time remaining, how many MBs downloaded out of how many - and the download speed at the moment, in KB/sec or MB/sec. There is also a button to pause, and a button to cancel.

In the beginning of the file download, the download speed may quickly reach certain value (say … 378 KB/sec … though in each case it is quite different). Then the speed starts dropping ... and dropping... Until it reaches some ridiculously low number (let's say … 67 KB/sec). If (if!) I do not disturb anything, from this point on, the speed dances around that low number till the end... that may come in a few hours... ;-)

(Mind you, my broadband connection has nothing to do with it; the size of my download pipe is always in excess of 20 MB/sec.)

However, through accident and experimentation, I discovered that I could immensely speed-up the above download “manually”, as follows:

“Technique 1” (some “technique”...): Observe the download speed, and as soon as the speed numbers start dropping off, pause the download, and then resume it after a second or so.

“Technique 2” (even faster): Each time the progress sliding bar stops moving, pause the download, and then resume it after a second or so.

Quite primitive, I admit, but it _works_. In fact, a download that would take two hours undisturbed, I can thus cut down to below ten minutes. The drawback, of course, is that it is “labour intensive” (… just imagine clicking the mouse like mad for ten full minutes...), and so, I shall not patent this “royal pain download accelerator”... :-)

But I do have questions:

1) My (uneducated) guess is that the above stop-start technique “fools” the YouTube server, which is perhaps deliberately set to give high speeds for small files only, and restrict the speed beyond certain quota. Is this indeed so?

2) And if it is so, do those “turbo-accelerators” work exactly as the above - “fooling” the server with start-stop, except that they do it automatically, instead of manually?

Thank you!

Johan

P.S. I now use another add-on for YouTube (while reserving the Download Helper for other web sites, where it is indeed useful), which is very fast by itself, and am wondering about it's design...

I have seen download behaviour as you have. My suspicion is that because many people bail on videos before they're completely done that YouTube might throttle downloads when the player is more than X% away from playing that portion. That's just a theory/guess, I have no real data to back that up. I honestly can't say how all acclerators work - they certainly could use many techniques to overcome various issues that impact download speed in addition to what's covered in this article.
Leo
11-Feb-2012
Terry Hollett
February 11, 2012 5:06 AM

From one perspective all web browsers have a mini accelerators built in called the cache. A copy of the web page being made on your computer first then loading up from your hard drive giving the impression of extra speed.

I believe this is how Google's web accelerator worked in the "dial-up" a days. And speaking of dial-up, when I had it and was still using IE I use to stop the download and start it up again if it dragged to slow. I would always get a little burst of speed when the download started.

I used download managers then that always promised faster downloads, they did not but still where very convenient when downloading through dial-up.

Glenn P.
February 12, 2012 9:01 AM

I do believe the "throttling" on UToob (heh! heh!) is simply their way of controlling the enormous bandwidth demands on their system. They send enough data quickly to build a good bit of buffered video, but then throttle back the speed to only slightly faster than the buffer cache empties. This way, YouTube's servers aren't overwhelmed (or at least, so they would hope!) with all the traffic they receive.

Engineer10388
February 13, 2012 1:54 PM

Leo, I agree that a download managers cannot increase your connection speed (in my case, DSL 6). But my experience with one, Internet Download Manager (by Tonec), is that it makes a substantial difference in effective download speed. The benefit can be dramatic when downloading a list of files, something that browsers typically do not handle well. The IDM interface shows that it processes several files at once and if your download is interrupted it will automatically resume when the connection is restored. Another nice feature is the ability to delay download(s) until a set time, such as when your computer(s) will not be using the internet connection for backup, streaming, etc. In short, I've found IDM to be a *very* worthwhile utility that has given rock-solid service since purchase about a year ago and I would buy it again in a heartbeat.

Henk
February 15, 2012 12:10 PM

@Johan. A few years ago I had the same 'stop and go' job for almost every download. The problem arose from a cable modem--new modem, no trouble anymore.

JMJM
February 16, 2012 3:13 AM

The initially fast, then slow download speed reported by Johan with YouTube is normal with any download in Firefox, in my experience. I must try the pause/resume trick. I find that Reload will kick (re)start a stalled web page download.

Comments on this entry are closed.

If you have a question, start by using the search box up at the top of the page - there's a very good chance that your question has already been answered on Ask Leo!.

If you don't find your answer, head out to http://askleo.com/ask to ask your question.