Helping people with computers... one answer at a time.

Slack space is unused, inaccessible space on a hard drive. I'll look at where it comes from, what you can do, and why it usually doesn't matter much.

I've heard quite a bit about disk slack space even though I don't completely understand that. From what I understand, it's the wasted space within the clusters (e.g. if you have a 2 KB file, the rest of the cluster is wasted) and that making smaller clusters (by repartitioning the drive) will make it smaller. What exactly is it, is that a big problem, and what can you do to reduce it in an already-partitioned system?

I wouldn't call it a big problem per se; most folks never notice any issues related to slack space. At worst, it simply means that your hard disk might run out of room a little sooner than it really needs to.

You're right in your analysis of what slack space is within clusters, but partitioning isn't really related and it's certainly not the answer.

I'll review how files are written to disk, what slack space is, and what you can do to minimize it.

If you even care, that is.

But first, we have to define a couple of terms.

Sectors

Physical disks are organized into sectors, which are blocks of bytes (or characters) of data. Disk reads and writes always operate on the entire sector. Changing even one byte in the sector means that the entire sector will be read, the byte changed, and the updated data written back to the sector.

"The only time you have an option of changing the cluster size is when you reformat the drive."

Most often, a sector will contain 512 bytes. (512 is a convenient power of two. Computers love powers of two Smile.)

Sectors are a characteristic of the hard disk hardware and are completely independent of operating system, file system, or just about anything else outside of the hard disk itself.

Clusters

Clusters are collections of contiguous sectors.

When you format a hard disk, part of that format determines the cluster size to be used when accessing that disk.

Clusters operate a lot like the sectors that they contain; each cluster is treated as single unit. When files are stored on a hard disk, they're allocated just enough clusters to contain the data.

Clusters may be configured to as little as one sector each to as many as eight, 16, 32, or even more.

Slack Space

Let's assume that you have a hard disk with 512 byte sectors and that each cluster is comprised of eight sectors. That means that each cluster will be 4096 bytes in length.

Now, let's create a file that's exactly one byte in length.

An entire cluster will be allocated to that file - all 4096 bytes will be removed from the available disk space and dedicated to this file.

One byte of that cluster will be used.

The remaining 4095 bytes would be what we call "slack space" - the unused, yet spoken-for space left over as an artifact of how file disk space allocation happens.

Multiply that over a lot of files and it might add up.

Why slack space usually doesn't matter

The most slack space that you'll ever see for any single file is one byte less than the disks cluster size. So the one-byte file that I used as an example above is a worst-case example. Assuming that file sizes are random, your average slack space per file will be roughly one half of the cluster size.

What that means is that the amount of disk space "lost" to slack is going to be roughly equal to the number of files that you have on your hard disk, times an average 2048 bytes.

When slack space might matter

The most common scenario where slack space might start to matter is if you have a lot of files that are smaller than half the cluster size and your hard disk happens to have a large-ish cluster size.

So if you're collecting a lot of files that are less than say 2048 bytes in size, then perhaps it might save you some disk space to choose a smaller cluster size.

Determining your cluster size

Honestly, the quickest way to determine a hard drive's cluster size in Windows is to fire up a Windows Command Prompt (be sure to run it as Administrator) and run the CHKDSK command:

Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

C:\Windows\system32>CHKDSK
The type of the file system is NTFS.
Volume label is NOTENQUAD.
...
lots of information
...
Windows has checked the file system and found no problems.

 976760000 KB total disk space.
 373357736 KB in 260775 files.
    107448 KB in 44648 indexes.
         0 KB in bad sectors.
    515420 KB in use by the system.
     65536 KB occupied by the log file.
 602779396 KB available on disk.

      4096 bytes in each allocation unit.
 244190000 total allocation units on disk.
 150694849 allocation units available on disk.

C:\Windows\system32>

"Allocation Unit" is synonymous with "cluster size". As you can see on this drive, my cluster size is 4096 bytes.

Changing the cluster size

Changing the cluster size is simple, yet it's a lot of work.

Reformat the drive.

The only time that you have an option of changing the cluster size is when you reformat the drive.

If you have an existing drive full of data, you'll have to copy off the data, reformat, and copy the data back.

If you're using the command line "Format", it's the "/A" parameter that allows you to specify the Allocation unit, or cluster, size. (Run "FORMAT /?" for a list of all options and allowed values.)

If, on the other hand, you use Windows Explorer, right-click the drive in question and click Format.... You'll get these options:

Format Options dialog

Note the "Allocation unit size" drop-down menu. Here, you can choose the allocation unit size that you wish the newly formatted drive to use.

The smallest is 512 - the current hardware sector size - cluster sizes smaller than this simply don't make sense. The largest will vary depending on the file system that you choose. The default for NTFS is typically 4096 and represents a reasonable compromise for disk-space usage, performance, and fragmentation as smaller clusters will typically result in more fragmentation.

In general, unless you have a specific reason to change it, I'd typically leave it at the default.

The good news is that as drives get bigger and cheaper, the impact of slack space becomes small enough to typically remain a non-issue.

† A new breed of hard drive is becoming more common place that uses "Advanced Format" sectors of 4096 bytes. This promises some significant performance gains, but it also presents some compatibility issues as well. Windows, for example, does not natively support 4k sectors, but rather relies on the drives supporting a 512 byte sector "emulation mode".

Article C4857 - June 27, 2011 « »

Share this article with your friends:

Share this article on Facebook Tweet this article Email a link to this article
Leo Leo A. Notenboom has been playing with computers since he was required to take a programming class in 1976. An 18 year career as a programmer at Microsoft soon followed. After "retiring" in 2001, Leo started Ask Leo! in 2003 as a place for answers to common computer and technical questions. More about Leo.

Not what you needed?

7 Comments
Mike
June 28, 2011 10:23 AM

I used to be quite concerned about slack space, but that was over 20 years ago, when HD's were typically 40MB, and even then it was wasted concern. Most files are more than 4KB (typical cluster size). So, in real life, slack space rarely wastes more than 1%-2% of HD space, if even that much.

Bob Fisher
June 28, 2011 11:25 AM

When you reduce the allocation size you increase the physical disk reads to obtain data which are extremely slow relative to the rest of the computer speed. If you reduce the allocation size from 4096 to 2048 and you need to read 8192 bytes, it will take you 4 physical reads versus 2 with the larger allocation size. Only reading 2048 or less bytes will be more efficient.

Steve Burgess
June 28, 2011 11:27 AM

Another area where slack space matters is in the world of computer forensics. We often find data left over in slack space from previous files that may be of importance to an investigation and reveal data that a user may have attempted to destroy.

Suppose you have written a file that fills its 4096 byte cluster, and then deleted it. The data isn’t gone until it is actually overwritten, and deleting doesn’t automatically overwrite the content of that cluster, as I’m sure many of your readers will know. Now suppose you come along and write to the same cluster the one-byte file you referenced in the article. We still have 4095 bytes of the previous file available for discovery of your nefarious deed!

An article called “Why Does Digital Forensics Matter to Me” that goes into a little more depth on this particular subject is here: http://www.burgessforensics.com/digital_forensics.php (I hope it’s okay to put this in).

Absolutely! Fascinating article. Thanks!
Leo
29-Jun-2011

B Gold
June 28, 2011 3:34 PM

Disagree with leo's explanation.
A file may be larger then 1 or many sectors or clusters.
only the last, fractional piece of a file resulkts in slack space. So the more large files you have, the less your average slack space per file. much less than Leo's statement of 'about half'.
The "worst case" scenario occurs only if most of your files are small files in their entirety - not just in their final sectors/clusters.

Mz
June 29, 2011 1:09 AM

disagree with leo that the only option changing the cluster size is reformat the whole drive - using True Image I changed my nearly full drive from 4k to 8k with no issues...but it took all night....copy off the data and copy them back when it's done is more safe though.

Mike
June 29, 2011 4:27 AM

@B Gold: Leo may have left out that files are typically larger than a single cluster, but he didn't say they would be mostly single clustered either; he only said that the amount of absolute space you lose is roughly half the cluster size multiplied by the amount of files you have.
It is true that the space wasted is relatively less if the average size of your files becomes greater because then there are more filled clusters per file and therefore fewer to waste.

My own slack space on my data partition is currently about 4.5GB, but relative to the total of 422GB that's still within the 1-2% margin. There's really no evading a little bit of waste, but there's no need to worry about it either.

Interociter Operator
July 6, 2011 6:51 PM

There was a product a few years ago that created a virtual drive out of hard drive space - STACKER. You could write as many small files as you wanted and it crammed them all together and managed their locations just as if it was a physical hard drive. It saved a ton of space on large drives with large cluster sizes when you had to save a lot of small files on it. It was fast too. What ever happened to that program?

Comments on this entry are closed.

If you have a question, start by using the search box up at the top of the page - there's a very good chance that your question has already been answered on Ask Leo!.

If you don't find your answer, head out to http://askleo.com/ask to ask your question.