Helping people with computers... one answer at a time.
We'll look at some of the specific information that you're making available to every website you visit, and briefly explain each.
What's my IP address?
As you probably already know your IP address is the "address" or logical (not physical) location of your computer or router connected to the internet.
There are plenty of sites that will tell you your IP address. In fact, any web site you visit can see it.
I'll do the same, but also try to explain, a little, exactly what it is you're seeing.
Your IP Address is:
This is the IP address of the device you have connected to the internet. That means that if you have a single computer it's likely to be the IP address of the computer itself. On the other hand, if you are behind a router, the IP address shown here is the internet-facing IP address of that router.
"Reverse DNS" on that IP address returns ec2-54-234-231-49.compute-1.amazonaws.com. If that's still just the IP address, there's nothing wrong with that. On the other hand, if that's an actual foo.bar.something.com kind of domain name, that information more often than not quickly and easily identifies your ISP.
Speaking of which, you often get even more information about the ISP that owns that IP address by visiting the ARIN whois database. ARIN, the American Registry for Internet Numbers, is the primary arbiter for the assignment of IP addresses and IP address ranges.
There are also various geo-locating services that will attempt to tell you where your IP address resides physically. I'm not going to link to any of them because they are notoriously inaccurate. Frequently, the only thing they get right is the country - even though they may claim much higher accuracy. (Mine, for example, typically is about 300 miles off.)
The User Agent is a relatively complex bit of information that identifies the browser that you are using as well as additional information related to your operating system and language. There are quite literally thousands of different browsers and spiders and other utilities for fetching web pages, and thus there are quite literally thousands of different possibilities for a User Agent string. It's also possible to configure some browsers or tools to lie - for example, it's possible to force FireFox to report a user agent string of Internet Explorer, to fake out websites that check for and only work with IE.
The referrer string is a little known but very interesting bit of information that's provided to web sites by most browsers. In short, if you click on a link to get to this page, the 'referrer' is the link back to the page you were on that had that link.
Depending on how you got to this page, it may or may not be set.
If you found this page via a search engine, things get even more interesting. Typically the URL of the page you were on in the search engine - the search results page that contained a link to this page - includes the search terms you were looking for that generated those results. That means that websites can know not only that you came from a search engine results page, but they're also typically also able to tell specifically what you were searching for.
There are other more obscure bits of information being passed around as well, but these are the big three; the ones that all websites get and that they might use in various ways.