How do I find out when a web page was written?

Definitions

First, I do need to be clear that Google has nothing, really, to do with this. It's just a way for you to find websites and web pages, and doesn't really factor into the "when was it written" question. So I'm explicitly not going to be talking about Google - or any search engine - at all.

Second, what we really care about here are not web sites, but web pages. A site may itself have a creation date, but in reality what we typically care about is the recency of a particular web page we're looking at.

The Best Source: The Page Itself

As silly as it sounds, the most authoritative source for the date of a web page is the web page itself. By this I mean that many pages include an "updated" or "posted" date somewhere on the page. Even here on Ask Leo! you'll see dates on all the articles:

In this example you'll see two: I place the most current "posted" date near the bottom of the article, below the related links.

The second is a reflection of the fact that the article I chose as my example had a major rewrite. So I choose to place the date of the rewrite as the posted date (highlighted here in the lower, red oval), and then include an explanatory statement about the rewrite, including its original date (in the upper, green oval).

There are, unfortunately, a bunch of problems with this:

• I could lie. There's nothing that forces these dates to be accurate.

• There's no standard location. Look above or below the main content of web pages, or perhaps in the web page footer, for the most common locations.

• They may not be there at all.

But in general, if the site takes the time to post a date with their content, that's where I'd head.

HTTP Headers

What most people want is some kind of magic date information that's somehow within the web page information - it must be there, we just can't see it.

Well, there is always a date returned, but it's not the date we want. When an HTTP request returns a web page the date that the response was generated is included. This is not the date of the page, it's (roughly) the date you requested the page.

Not helpful.

There is sometimes an additional date returned called "Last Modified", which is intended to reflect the date that the page being requested was last altered.

Once again, there are several problems with this approach:

• It's not required. In fact, in researching this issue, I note that my server does not send Last Modified information when you fetch a page.

• There's no standard as to exactly what it means.

• Typically it means the last date (and time) that the file you're accessing was altered - but that can often have no relationship to when the content it contains was written. For example, on my site a page is "altered" every time someone leaves a comment as the page is updated to contain the new comment. That is completely unrelated to when the article itself was written.

• It could lie.

So the closest technological resources we have is woefully inadequate.

(If you want to see the headers that are returned with web page requests, Firefox has several addons that will do so. My favorite is Firebug - actually a fairly extensive web debugging addon. Another might be "Live Headers", an addon that simply displays headers. I'll warn you that you'll be heading into some fairly geeky territory, though. )

History

OK, I lied: I'll mention Google one more time.

Google, and all other search engines like it, could track historical changes to pages as they periodically spider the internet. If one week a page appears, and the next week it changes, it seems like search engine spiders could track this activity.

To the best of my knowledge they do not. Or if they do, they don't typically make the information available.

The Internet Archive, on the other hand does exactly that.

Using the Internet Archive's "Wayback Machine", I can actually view web pages "as they were" at some point in the past. As long as the Internet Archive had spidered and captured that web page on that date in the past, that is.

Sadly, the Internet Archive also has some serious limitations:

• It's spotty - even within a site not all pages on that site may be included.

• It's spotty - not all sites are included. In fact, webmasters can actually request that they not be included.

• It's spotty - not all dates are included. The Internet Archive's spider checks "periodically" at what appears to be a rate of every few weeks. Changes occurring faster than that are not captured.

• It may not be current. They state that it may take up to 6 months for pages to appear. The Ask Leo! home page, for example, is current on Internet Archive only through June of 2008.

But even with all those limitations, it can be a useful piece of data for researching when, approximately, a web page changed.

If it's in the archive, of course.

The reason archiving of this sort is so challenging is simply the sheer quantity of data involved. An ideal archive would keep an entire copy of the entire world wide web every so often. That's more data than can be reasonably managed.

Apparently It's Timeless

Combining those approaches can often get you interesting information, but as we've seen: each approach has some serious limitations.

In the end the answer remains no. There's no definitive way to determine when a web page - or its content - was created.