Helping people with computers... one answer at a time.
Slack space is unused, inaccessible space on a hard drive. I'll look at where it comes from, what you can do, and why it usually doesn't matter much.
I've heard quite a bit about disk slack space even though I don't completely understand that. From what I understand, it's the wasted space within the clusters (e.g. if you have a 2 KB file, the rest of the cluster is wasted) and that making smaller clusters (by repartitioning the drive) will make it smaller. What exactly is it, is that a big problem, and what can you do to reduce it in an already-partitioned system?
•
I wouldn't call it a big problem per se; most folks never notice any issues related to slack space. At worst, it simply means that your hard disk might run out of room a little sooner than it really needs to.
You're right in your analysis of what slack space is within clusters, but partitioning isn't really related and it's certainly not the answer.
I'll review how files are written to disk, what slack space is, and what you can do to minimize it.
If you even care, that is.
But first, we have to define a couple of terms.
•
Sectors
Physical disks are organized into sectors, which are blocks of bytes (or characters) of data. Disk reads and writes always operate on the entire sector. Changing even one byte in the sector means that the entire sector will be read, the byte changed, and the updated data written back to the sector.
Most often, a sector will contain 512 bytes. (512 is a convenient power of two.
Computers love powers of two 
.)
Sectors are a characteristic of the hard disk hardware and are completely independent of operating system, file system, or just about anything else outside of the hard disk itself.
Clusters
Clusters are collections of contiguous sectors.
When you format a hard disk, part of that format determines the cluster size to be used when accessing that disk.
Clusters operate a lot like the sectors that they contain; each cluster is treated as single unit. When files are stored on a hard disk, they're allocated just enough clusters to contain the data.
Clusters may be configured to as little as one sector each to as many as eight, 16, 32, or even more.
Slack Space
Let's assume that you have a hard disk with 512 byte sectors and that each cluster is comprised of eight sectors. That means that each cluster will be 4096 bytes in length.
Now, let's create a file that's exactly one byte in length.
An entire cluster will be allocated to that file - all 4096 bytes will be removed from the available disk space and dedicated to this file.
One byte of that cluster will be used.
The remaining 4095 bytes would be what we call "slack space" - the unused, yet spoken-for space left over as an artifact of how file disk space allocation happens.
Multiply that over a lot of files and it might add up.
Why slack space usually doesn't matter
The most slack space that you'll ever see for any single file is one byte less than the disks cluster size. So the one-byte file that I used as an example above is a worst-case example. Assuming that file sizes are random, your average slack space per file will be roughly one half of the cluster size.
What that means is that the amount of disk space "lost" to slack is going to be roughly equal to the number of files that you have on your hard disk, times an average 2048 bytes.
When slack space might matter
The most common scenario where slack space might start to matter is if you have a lot of files that are smaller than half the cluster size and your hard disk happens to have a large-ish cluster size.
So if you're collecting a lot of files that are less than say 2048 bytes in size, then perhaps it might save you some disk space to choose a smaller cluster size.
Determining your cluster size
Honestly, the quickest way to determine a hard drive's cluster size in Windows is to fire up a Windows Command Prompt (be sure to run it as Administrator) and run the CHKDSK command:
Copyright (c) 2009 Microsoft Corporation. All rights reserved.
C:\Windows\system32>CHKDSK
The type of the file system is NTFS.
Volume label is NOTENQUAD.
...
lots of information
...
Windows has checked the file system and found no problems.
976760000 KB total disk space.
373357736 KB in 260775 files.
107448 KB in 44648 indexes.
0 KB in bad sectors.
515420 KB in use by the system.
65536 KB occupied by the log file.
602779396 KB available on disk.
4096 bytes in each allocation unit.
244190000 total allocation units on disk.
150694849 allocation units available on disk.
C:\Windows\system32>
"Allocation Unit" is synonymous with "cluster size". As you can see on this drive, my cluster size is 4096 bytes.
Changing the cluster size
Changing the cluster size is simple, yet it's a lot of work.
Reformat the drive.
The only time that you have an option of changing the cluster size is when you reformat the drive.
If you have an existing drive full of data, you'll have to copy off the data, reformat, and copy the data back.
If you're using the command line "Format", it's the "/A" parameter that allows you to specify the Allocation unit, or cluster, size. (Run "FORMAT /?" for a list of all options and allowed values.)
If, on the other hand, you use Windows Explorer, right-click the drive in question and click Format.... You'll get these options:
Note the "Allocation unit size" drop-down menu. Here, you can choose the allocation unit size that you wish the newly formatted drive to use.
The smallest is 512 - the current hardware sector size† - cluster sizes smaller than this simply don't make sense. The largest will vary depending on the file system that you choose. The default for NTFS is typically 4096 and represents a reasonable compromise for disk-space usage, performance, and fragmentation as smaller clusters will typically result in more fragmentation.
In general, unless you have a specific reason to change it, I'd typically leave it at the default.
The good news is that as drives get bigger and cheaper, the impact of slack space becomes small enough to typically remain a non-issue.
•
† A new breed of hard drive is becoming more common place that uses "Advanced Format" sectors of 4096 bytes. This promises some significant performance gains, but it also presents some compatibility issues as well. Windows, for example, does not natively support 4k sectors, but rather relies on the drives supporting a 512 byte sector "emulation mode".
Article C4857 - June 27, 2011
Leo A. Notenboom has been playing with computers since he
was required to take a programming class in 1976. An 18 year career as a programmer at Microsoft soon followed.
After "retiring" in 2001, Leo started Ask Leo! in 2003 as a place for answers
to common computer and technical questions. More about Leo.
Another area where slack space matters is in the world of computer forensics. We often find data left over in slack space from previous files that may be of importance to an investigation and reveal data that a user may have attempted to destroy.
Suppose you have written a file that fills its 4096 byte cluster, and then deleted it. The data isn’t gone until it is actually overwritten, and deleting doesn’t automatically overwrite the content of that cluster, as I’m sure many of your readers will know. Now suppose you come along and write to the same cluster the one-byte file you referenced in the article. We still have 4095 bytes of the previous file available for discovery of your nefarious deed!
An article called “Why Does Digital Forensics Matter to Me” that goes into a little more depth on this particular subject is here: http://www.burgessforensics.com/digital_forensics.php (I hope it’s okay to put this in).
29-Jun-2011
Posted by: Steve Burgess at June 28, 2011 11:27 AM
Disagree with leo's explanation.
Posted by: B Gold at June 28, 2011 3:34 PMA file may be larger then 1 or many sectors or clusters.
only the last, fractional piece of a file resulkts in slack space. So the more large files you have, the less your average slack space per file. much less than Leo's statement of 'about half'.
The "worst case" scenario occurs only if most of your files are small files in their entirety - not just in their final sectors/clusters.
disagree with leo that the only option changing the cluster size is reformat the whole drive - using True Image I changed my nearly full drive from 4k to 8k with no issues...but it took all night....copy off the data and copy them back when it's done is more safe though.
Posted by: Mz at June 29, 2011 1:09 AM@B Gold: Leo may have left out that files are typically larger than a single cluster, but he didn't say they would be mostly single clustered either; he only said that the amount of absolute space you lose is roughly half the cluster size multiplied by the amount of files you have.
It is true that the space wasted is relatively less if the average size of your files becomes greater because then there are more filled clusters per file and therefore fewer to waste.
My own slack space on my data partition is currently about 4.5GB, but relative to the total of 422GB that's still within the 1-2% margin. There's really no evading a little bit of waste, but there's no need to worry about it either.
Posted by: Mike at June 29, 2011 4:27 AMThere was a product a few years ago that created a virtual drive out of hard drive space - STACKER. You could write as many small files as you wanted and it crammed them all together and managed their locations just as if it was a physical hard drive. It saved a ton of space on large drives with large cluster sizes when you had to save a lot of small files on it. It was fast too. What ever happened to that program?
Posted by: Interociter Operator at July 6, 2011 6:51 PM