Helping people with computers... one answer at a time.
It's not uncommon to use a web browser other than Internet Explorer. What you may not realize is that you're still using IE for other things.
It's needed in more than a "minimal way", so not only is it best not to delete it, it's critical that you do not.
Portions of what you and I consider Internet Explorer are actually part of Windows itself and would cause portions of Windows to fail (more than just the Windows Update website). In fact, it could quite possibly cause other applications that depend on the functionality that those parts of IE-in-Windows provides to fail as well.
The primary job of Internet Explorer, like most web browsers, is to display HTML encoded web pages.
The problem is that it's not just web pages that are encoded in HTML these days. Many applications use HTML as part of their user interface, their documentation, or their online help system.
To that end, the core HTML rendering engine in Internet Explorer is actually functionality used and provided by Windows for other applications to use.
That means that you can't just yank it out.
In fact, if you try by going to Control Panel, Add/Remove Programs, Turn Windows features on off, and de-selecting Internet Explorer, Windows will inform you of the likely consequences:
So, I guess that means that you could turn it off ...
But you really don't want to.
As you suggest, you can consider tightening Internet Explorer's security.
... uninstall or disable addons.
... turn the security level on each of IE's security zone's to High.
... disable third-party extensions.
There are probably more ways.
Those are all fine things to do ... until, as a side effect, they break one of the other applications that uses Internet Explorer's functionality in Windows.
I'm not discouraging it - it's easy and safe to revert a security setting if you find out that something breaks.
I'm just not encouraging it as I believe that it's a fair amount of work for not much additional return.
Even if you don't use Internet Explorer as your browser, you can probably see by now that the components that make up IE are also frequently used elsewhere.
Because they are still being used, it's important that they be kept up to date.
That means if Windows Update offers you an update related to Internet Explorer, don't dismiss it out of hand because you don't use IE.
While you don't need to take the version-to-version updates until something else forces the issue, it's important to continue to take Internet Explorer security updates even if you don't use Internet Explorer.
My approach is a little different.
I do nothing.
Or, put another way, I do nothing out of the ordinary; I keep IE up to date, but I don't go in and tweak any of its settings for additional security.
I use IE very infrequently but occasionally, I just have to - as I suspect that you do as well.
So I keep it in a runable, usable state.
Not using it as your browser actually avoids most of the exploits that are out there. Certainly not all, but this is where I look at the work/risk tradeoff. It's just not worth it to me to take extra steps.
Besides, recent versions of IE continue to improve, both in terms of performance and security. While I might use another web browser most of the time, having IE lying around doesn't really impact my overall vulnerability by that much, in my opinion.