Helping people with computers... one answer at a time.

Programs that identify identical files can be useful, but simply deleting the duplicates found can cause more harm that good.

Is it safe to delete duplicate files found by duplicate file finder programs?

I'm probably going to get some flack for this one, but my answer is a very strong NO.

Let me explain why.

In short, installing programs in Windows is a mess.

Over the years programs have gone from attempting to share large numbers of components, to keeping separate copies of often identical files in order to protect from unexpected upgrades that can sometimes cause software to fail.

Right now, most programs are somewhere in between. Some modules are shared, and some are not. The "mess" is that there's no consistent rule, and applications are free to choose what they do and do not share, as well as what they do or do not rely on and install themselves.

Let's use an example.

Program "A.EXE" and program "B.EXE" are both statistical packages, and they both rely on a fairly large library of mathematical functions that are packaged into "STATS.DLL".

"In short, installing programs in Windows is a mess."

Scenario 1:

When you install A.EXE, it installs STATS.DLL into a place where anyone can use it. Then later, when you install B.EXE, it sees that STATS.DLL is already on your machine, so it has no need to install it. B.EXE just uses the STATS.DLL that's already there.

Sounds fantastic in principal since disk space is saved, and there's only one STATS.DLL to keep track of.

Then you take an update to A.EXE, which includes a new version of STATS.DLL. The update dutifully replaces the older copy of STATS.DLL with the new, and the new A.EXE is happy.

And your old B.EXE breaks.

Why? Well, to be clear, it's not supposed to. But all too often it does, because something about the newer STATS.DLL didn't take into account the way that B.EXE was using it. Could be either components fault, but the fact is that B.EXE doesn't work with the newer STATS.DLL.

You're stuck, until B.EXE is revised to work with the newer STATS.DLL. If B.EXE is never updated, you're basically screwed.

Scenario 2:

So the makers of B.EXE and A.EXE get tired of this situation where they can't update their software without affecting each other. So they decide to take a different approach.

Now, when you install A.EXE, it installs STATS.DLL into a private place where only A.EXE can use it. Later, when you install B.EXE, it does the same thing, installing another copy of STATS.DLL where only it can use it.

Problem solved, right? Now when A.EXE is update, it only update's its copy of STATS.DLL and B.EXE is unaffected.

But of course you have two copies of STATS.DLL on your machine. In fact, you may have two identical copies of STATS.DLL on your machine.

And deleting either one would be a big mistake.

So here's the bottom line:

Yes, it's safe to delete some of the duplicate files that your duplicate file finder may identify. It's just not easy to tell which ones. And without knowing, it's possible very possible you'll break an application, or worse, your operating system.

My advice: don't.

If you're looking to free up space on your hard drive, use tools that show you where the most space is being used and target your efforts to get the biggest bang for your efforts.

But in general, unless you know what you're doing, leave duplicate files alone.

Article C3072 - July 1, 2007 « »

Share this article with your friends:

Share this article on Facebook Tweet this article Email a link to this article
Leo Leo A. Notenboom has been playing with computers since he was required to take a programming class in 1976. An 18 year career as a programmer at Microsoft soon followed. After "retiring" in 2001, Leo started Ask Leo! in 2003 as a place for answers to common computer and technical questions. More about Leo.

Not what you needed?

29 Comments
gilles
July 3, 2007 9:06 AM

VERY GOOD ADVICE AS USUAL.ABOUT A YEAR AGO I USED A LITTLE PROGRAM TO GET RID OF DUPLICATE FILES,IT DID THAT ALLRIGHT AND AT THE SAME TIME IT GOT RID OF XP AND I HAD TO REINSTALL,SO LISTEN TO LEO.

Terry Hollett
July 7, 2007 5:35 AM

I got a call from a guy who couldn't get MSN Messenger to run. I couldn't either even after reinstalling it. Also I noticed the adware program wouldn't run. I downloaded that version and even that wouldn't run.

An antivirus scan showed no infection. The only clue was when trying to run Lavasoft Ad-Aware it wouldn't but there was a error message concerning the file RICHED20.DLL. I assumed it was corrupted/missing - did a search and found 4 instances of the file on the computer.

I decided to download a new copy of the internet. They didn't have the install CD and I wasn't sure about the copies on the system. After downloading I wasn't sure where to put it. I chose SYSTEM32 folder even though there wasn't a copy there. Problem solved.

I had a similar problem with WMP.DLL from Media Player 10. Found 4 instances of the file on my hard drive and decided to use one of them. It worked by coping the file from one folder (can't remember the name right now) to the SYSTEM32 folder where again the file was missing. It worked. Duplicate files aren't always bad.

www.geocities.com/terryhollett2003/feb202007.htm

Jerry
July 7, 2007 7:41 AM

On March 22, 2007 I deleted all duplicate files, and my system crashed as a result. Leo is giving good advice.

Cat Moves
July 7, 2007 9:45 AM

For some now, I 've, almost religiously, been running CCleaner, then Disc Cleanup, then Drive Cleanup before turning off my computer. I wanted to know how it looked once, so after performing my usual shut down cleaning, when I next turned it on, I ran the same programs. There's a surprise. It seemed that all those nasties (or most of them at least) were back. I had done no surfing, no going on online, nothing. What's going on here? Am I wasting my time doing these chores?

Ziggie
July 9, 2007 11:34 AM

I've learned this lesson the hard way. Once I ran one of these duplicate finders to clear up some duplicate files (lean = fast, right?). When I got done ALL of my shortcut files had disappeared. My desktop was blank, my start menu was blank, my quick launch was blank. I've decided since then that duplicate files aren't that bad....

Linda
July 20, 2007 7:26 PM

I agree that deleting duplicates files could be a problem IF you're not looking at the duplicates before you delete them. Checksums are important when integrating it into a duplicate file program.

Also, I'm still wondering how someone could lose their operating system without knowing it before they click.......hmmmmmm

john290
February 26, 2008 9:44 AM

I agree with all of you! but i am sys admin and i create lots of backup... so to organize this we use duplicate finder it help me a lot!

http://www.ashisoft.com

Kristian
April 7, 2008 7:21 AM

As Leo and many other in this thread says it is not safe to remove all duplicate files. I would recomend only to remove user files (digital photos, music, documents etc) and carefully check every file before removing it and not use any programs that automaticaly removes clones without your control. The problem is not only to find the duplicate files but to select which ones to remove.
You could try using our software Clone Tools that will give you control of exactly what you are doing by displaying all clones in a file browser. Windows system directories are exlcuded from the duplicate scan by default.
http://www.pilotman.com

Max McCord
July 31, 2008 3:17 AM

Well Leo, You shot me down...sorta. I discovered your site while seeking an efficient solution to a duplicates problem (some unknown application or some action on my part has resulted in vast numbers of duplicate files, primarily shortcuts, on my Vista system. It would take many hours to remove them all individually). My problem is not primarily a duplicate program removal issue, but you have convinced me to be extra cautious. I have encountered problems such as you describe in the past, sometimes with disastrous consequences. Thanks for reminding me of the potential problems.

Allan
October 11, 2008 8:36 PM

Never delete files from the Windows directory
Only delete files you know are safe to delete

I use Directory Report to find dups
http://www.file-utilities.com/downloads/wdir.zip

femma
October 19, 2008 4:10 PM

hi leo,
my neighbor put my photos in my desktop 900PLUS CAN I DELETE AND HOW ALSO AN ICON APPEARS AND I CAN NOT DELETE SAIDS TMP FILE THANKS FOR YOUR HELP

Charley
January 23, 2009 9:16 PM

Wish I had read this like 3 days ago. Downloaded a duplicate file finder, one highly rated by all review sites and consumer reviews. I did some duplicate file removing for a couple of days without a hitch. As confidence grew I decided to give it a real workout and check Windows and all it's subfolders for duplicates. As you can imagine there were over a thousand duplicates listed, some with five duplicates of the same file, or so I thought. Diligently began checking file against file, tried to get an answer as to which would be safest to delete, ie oldest, newest, longest, shortest, but found no information. Remembered that a program from Toni Arts suggested keeping oldest files, but I have been messing with this computer for so long it didn't help much. I told program to keep all hard links from being scanned and thought now I could go to it. Well here I sit 3 days and a gazillion hours later and am just now able to use the computer again. The questioer asking about couldn't I see operating system and stuff was being taken, of course not. I wasn't even deleting the files just sending them to recycle bin temporarily keeping location information with each file. Guess what, started the process and about three minutes into it the screen went black, computer tried to reboot and repeatedly tried to reboot until I stopped it. Could not run any of the boot options like safe boot or last good startup, nothing. Just kept saying starting Windows than shutdown and restart. Had to use Windows CD to try and start computer, but it couldn't repair my Windows installed on computer and had to load a new copy of Windows to the computer. Three days later I am happy to say everything is back to where it was prior to my trying to get rid of some stupid duplicate files. Guess Stupid is really what Stupid does, eh? Word to the wise, unless you are single, jobless, and bored; live with the duplicates and buy an external hard drive if you want quick extra disk space. Worth the money, but more importantly worth not having to go through the frustration and time of all the above.

Glenn P.
January 27, 2009 11:09 AM

To quote the tuna fish commercial, "Sorry, Charley." :)

Folks, Leo has said this MANY times before:

BACKUP your &@^($% computer before trying anything so radical! (Like, "Duh...?")

I use BootItNG, and I would make a full backup before doing anything like Charley did. It boots up before Windows does, and saves its files to an external drive, so if need be -- and provided only that the C: drive itself is still physically intact -- I can restore my computer to its full pristine Before-Disaster-State even if my entire Win XP Pro operating system is completely hosed.

It's the best insurance you could ever have, folks: an External Drive, BootItNG, and You. Mutato nomine de te fabula narratur.

Bob D.
January 28, 2009 8:40 AM

I agree with the poster who advised buying a large disk. Disks have long been cheaper than lost files. Also, get an un-deleter program. If you un-delete before the deleted file's disk blocks are re-used, you can save much grief. I've used Vcom's Fix-It un-deleter a couple of times in the past few years, much to my relief. And please excuse my repeating the often-heard advice "BACK UP YOUR DISKS!" If you have a machine reserved for hacking, back up a full-disk image from your real-work machine, and then restore it to the hack machine, to be sure the cycle works. Then delete files from your real machine, if you like stress and hassle.

BRANDON
March 11, 2009 8:33 PM

If it's space you're looking for and duplicate files in your windows directories is the problem (especially dll's) a "clean install" is your best bet if you want to avoid headaches :)

Kim
June 21, 2009 2:05 PM

i found the safest duplicate file finder to find and remove duplicate files...

http://www.duplicate-finder-pro.com

snail
June 23, 2009 5:53 PM

I agree with keeping out of system files and application files but I do need a streamlined setup for sifting through would be duplicates or candidates of redundant files.
For finding my duplicate files I like to have open on a large monitor a folder where each file is located(A original, and B...Z possible backups/redundant files).
I like to display as much info as possible but sometimes I compare folders and have not been able to display the size of folders without pressing Alt + enter(properties). There is other info I would like to see in order to more quickly glance at attributes and thumbnails to compare.
However, had I set out to organize user-made files into something like follows, I wouldn't have these problems: 1)Main hard drive, created file
2)backup(sequentially, noting date of amendments or revisions, deletions, moving, et cetera
3)reference for further organization, such as topics of commonality(use shortcuts??)

ORGANIZE

Allan Cass
July 19, 2009 12:22 PM

You can try this FREE tool that was recently featured on PCMagazine and ZDNet it has protection for system files:
http://www.mindgems.com/products/Fast-Duplicate-File-Finder/Fast-Duplicate-File-Finder-About.htm - Fast Duplicate File Finder
Fast Duplicate File Finder will help you find fast all duplicate files in a folder and its sub folders.
The applications will compare the content of your files so it will find duplicates even if they are using different file names.
It uses fast binary comparison algorithm and has internal preview supporting a lot of image, video, music and text file formats.

DeRose
November 16, 2009 12:50 AM

There are some programmes for XP to find duplicate files. I wanted to delete music duplicate file I used the programme Delete duplicate files for such aim. It helped me. This program is not only for the music, but for all files. And now it is more free place on my PC :)

Janis
November 22, 2009 6:10 AM

help - since deleting all my duplicate files through software I downloaded I have no start bar or icons on my desktop. I suspect, I've done more damage than that. The only way I can get online is through taskbar. Is there anyway to restore my computer to the day before I did this. I can get into dos through my taskbar. I am away and cannot get to my original Windows XP discs. Thanks, Jf

Liam
January 7, 2010 11:32 AM

You can try these duplicate finders. They are really safe as they include protection for system files.
http://www.find-duplicates.com

Jade
March 22, 2010 4:23 AM

Ya its true but you should use a tool which ,remove duplicate files which are actual waste files, that should not effect after deleting them, to remove actual duplicate files i use a tool from Ashisoft.

Download Free : http://www.ashisoft.com

Glenn P.
June 29, 2010 11:01 AM

Ever hear of "backups"? I often intentionally place duplicates of important files in several places, so that if one is corrupted in one place on the drive, I have another copy.

Not all duplicates are "evil" even when their removal doesn't break anything.

James
July 1, 2010 3:26 AM

For disk space management, I can recommend TreeSize, which I've been using for over ten years. There is a free version at http://www.jam-software.com/freeware/index.shtml#treesize. Of course, for a few dollars more you can get a lot more functionality from TreeSize Personal.

Ron
July 4, 2010 6:32 PM

I'm with you. Deleting dupes takes thought. The first thought being, MAKE A FULL BACKUP BEFORE STARTING!

Second thought, NEVER, EVER, run a dupe finder in auto delete mode.

Third thought, stay away from system files/folders.

Next, be very very careful when deleting application related files.

Next, deleting dupes is safest when dealing with user files, until you delete the wrong copy of one of your data files.

Finally, when in doubt, DON'T DELETE IT!

Instead of deleting dupes, I often use Windows file compression on Folders for apps that I don't use often.

One place I do find dupe finder tools handy is when apps install support for multiple languages that I'll never use (ie help files, EULA's etc). The files typically have the same name even though the contents are in different languages. Them I gladly nuke (and search for folders with similar names, typically 4 digit numbers). Granted, they don't amount to much space freed up, but they also represent larger blocks that are freed up. I'll take any advantage I can get.

Carlos Coquet
July 24, 2010 2:19 PM

Yeap. That is PRECISELY what I dislike in Windows, the lack of modularity. The original IBM PC's modularity was, in great part, the reason for its success. (You had a barebones box and just plugged into slots whatever you wanted / needed.) Windows sneaked into prominence in spite of being built on the concept of interdependence. Anything you install or anything you uninstall can make changes to the Registry (what I call the cancer in Windows) and remove modules needed by other programs causing anything from annoying messages to a total inability to run Windows or some other program you own, totally unrelated to the one you installed or removed.
Incidentally, this explains why most Windows programs cannot be just copied from one machine to another by just copying the directory. If the developer chose to use the Registry or shared modules, (amazingly, most do) the program will not work. Also explains why you generally cannot remove a program from Windows by just deleting its directory either. For one thing, you may leave all sorts of trash in the Registry and / or shared modules area.
Every release that has come since Windows 3.1, I have hoped to see a move away from this complete interdependence but it has yet to happen.

Janet
January 9, 2012 3:35 AM

PLease advise I appear to have hundreds of duplicate GIF Images, for example: at least 20 of the following: cf2_09-03 32.6 KB (33,445 bytes, Toshiba Configfree Gif Image, there are many more, can I delete the duplicates? thank you. Kind Regards

delete identical files
December 24, 2012 9:37 PM

I'm with you. Deleting dupes takes thought. The first thought being, MAKE A FULL BACKUP BEFORE STARTING!
Second thought, NEVER, EVER, run a dupe finder in auto delete mode.
{URL Removed}

shaimaa
January 30, 2013 2:08 AM

i have used http://DuplicateFilesDeleter.com and it helped me alot, hope it does for you

i guess it is safe to delete file , it didnt harm my computer, am not promoting but i am sharing my experience

Comments on this entry are closed.

If you have a question, start by using the search box up at the top of the page - there's a very good chance that your question has already been answered on Ask Leo!.

If you don't find your answer, head out to http://askleo.com/ask to ask your question.