Helping people with computers... one answer at a time.
That's really a much broader question than I can answer here, however, my recent experiences both being hacked, and building out a new server to replace the hacked one, are worth recording. Not only might you find it a helpful laundry list of idea for securing your server, but you might also have some ideas and experiences to share.
So these are the steps I've taken to "harden" or more tightly secure a newly built out Linux server. These steps are intended to reduce the probability the server being hacked into again.
In this particular case our brand new server is built out with Redhat Enterprise Linux, with the cPanel management interface. A 2.0Ghz Xeon, with 512Meg RAM, and a 54gig hard drive.
The goal at this step is actually fairly simple: start out with a popular, current, and supported, version of the OS.
Step one was to login to Web Host Manager (the server management front-end for cPanel), and run their configuration wizard. The only things that required non-default action were:
After the wizard was complete, these additional settings were performed manually:
Also ran Update Server Software and Update System Software within WHM to update as many components as reasonable.
The original configuration as delivered by the server farm:
WHM 10.1.0 cPanel 10.2.0-R82
RedHat Enterprise 3 i686 - WHM X v3.1.0
Linux kernel version: 2.4.21-27.ELsmp
Since that's not the latest kernel at the time of delivery, and cPanel will not update the kernel automatically, I was advised to update. I upgraded the kernel to 2.4.21-32.0.1.ELsmp, which went smoothly.
One of the areas for almost immediate security tweaking is remote access. If you've ever examined a server's logs, you know that they're regularly probed with various types of attacks. There are several modifications to render those attacks pointless, as well as secure remote access in general.
SSH, or secure shell, is the remote command-line access mechanism of choice. It's single biggest claim to fame is simply that everything is encrypted, though it includes many more features as well.
I first created a couple of non-root login accounts, and installed RSA public keys for eventual RSA/public key-only logins. (Also created end-user instructions for creating, and using, SSH key pairs.)
I then made the following changes to /etc/ssh/sshd_config:
I disabled telnet, as it allows passwords to be transmitted in the clear, and is a known security risk.
Normally at this point I would also disable the FTP service for the same reasons as Telnet - it's a known security risk. For reasons I didn't bother to investigate further, cPanel either complained repeatedly via email if the service was stopped, or it would restart it, depending on the technique I was attempting to use. I gave up on this and left the FTP service running, but blocked at the firewall, discussed below.
The /tmp directory is an interesting and apparently common place for hackers to exploit to breech a system. One approach to securing it without breaking its functionality is to create it as a new partition that disallows program execution. Where in the past a hacker could use an exploit to place a file in /tmp and then execute it, the ability to execute is now removed by creating a secure /tmp partition.
Installed APF which is, essentially, an intelligent front end to the built in Linux firewall, iptables. I found a good how-to, though I did not yet configure the antidos (denial of service) component. The one change that I did make to the default configuration is that I disabled or blocked all ports that we know are not explicitly used for something specific. For example, this is where I blocked all FTP access. I also whitelisted my own static IP address, and that of the client.
A few additional tweaks and settings ...
Even with all the tools and updates in place, there's still on-going work and vigilance required.
I haven't implemented these, but they've been recommended, and they're on the list to evaluate at some point.