Looking at America Online…Literally

AOL Research recently released a collection of search queries that AOL users had entered from March 1, 2006 – May 31, 2006. Attached to the data was the following description of the file contents: This...

Approximate Read Time: 4 Minutes

AOL Research recently released a collection of search queries that AOL users had entered from March 1, 2006 – May 31, 2006. Attached to the data was the following description of the file contents:

This collection consists of ~20M web queries collected from ~650k users over
three months. The data is sorted by anonymous user ID and sequentially arranged.
The goal of this collection is to provide real query log data that is based on
real users. It could be used for personalization, query reformulation or other
types of search research.
The data set includes {AnonID, Query, QueryTime, ItemRank, ClickURL}.
AnonID - an anonymous user ID number.
Query  - the query issued by the user, case shifted with
most punctuation removed.
QueryTime - the time at which the query was submitted for search.
ItemRank  - if the user clicked on a search result, the rank of the
item on which they clicked is listed.
ClickURL  - if the user clicked on a search result, the domain portion of
the URL in the clicked result is listed.

In other words, this data trove is pretty much the same type of materials that Google fought vigorously to keep out of the hands of the U.S. government. While AOL has since removed the data from its website, copies of the data file are still floating around the Internet. However, instead of downloading and processing the raw data by yourself, which is no small feat considering the amount of data involved, I’ve come across a website — appropriately called AOL Search Database — that lets you search through the AOL search query data by User ID, search keyword or website result.

Here’s a quick review of the site.

Easy to Use. Beats setting up a database and importing 36 million lines of data.

But…

No Free Text Search. Keyword search is not the same as free text search. If I search for gps, no results are returned because gps is not on the keywords list. That’s restrictive…

AND/OR. When I search for stanley cup on Google, it assumes that I want to search with all of the words; i.e., an AND operator between the search terms. In contrast, AOL Search Database assumes an OR operator between the search terms. So, stanley cup returns stanley cup, as well as morgan stanley, stanley cleaners and stanley furniture, which aren’t exactly relevant to my search query.

With those limitations in mind, let’s see what this data reveals.

A lot of useful data may be extracted from the AOL Search Database. For attorneys looking to build their Internet presence, understanding exactly how individuals search for attorneys will allow you to tailor your website to most effectively reach prospective clients.