The Rise of Search Engines: A Brief History of the Web’s Most Essential Tool

The search engine is arguably one of the most important inventions of the 20th century. Without it, we’d be forced to wander through the depths of the Internet without a guide, unable to find what we seek without knowing the exact site URL. The search engine is our virtual compass; it’s our primary way of navigating the wilds of the World Wide Web. Without it, we’d all be lost.

Considering the search engine is something so vital to our everyday life, we know surprising little about the tool’s origins. Curious to know how Google, Bing and Yahoo! came about? Here’s a brief history of the almighty search engine.

Meet Archie…and Jughead…and Veronica

The first basic search engine was created by three computer science students at McGill University in 1990. The tool was named Archie (a play on the word “Archive”). Archie created a searchable database of public file names on FTP (File Transfer Protocol) sites. Since the tool was launched before the Internet took off on a large-scale basis, the database was small enough to be searched manually.

After the Gopher protocol was introduced, Archie evolved into two further search programs named Veronica and Jughead (yes, after the Archie comic book characters). However, we still didn’t have a web-based search tool for the World Wide Web…yet.

Dark Ages: The Pre-Google Era

Though the World Wide Web (and with it, HTML and HTTP) was created in 1990, it wasn’t made publicly available until 1993. Soon after its launch, Matthew Grey of MIT created the very first web robot; this first bot was then used to search and index the Web through a program called “Wandex.”

A year later, we had the Wanderer: the world’s first full text crawler. Most consider the Wanderer as the origin for the modern search engines we know today — the search engine Neanderthal to Google’s Homo sapiens. 

The years that followed resulted in a slew of search engines the Web has long since forgotten: Lycos (the world’s first commercial engine), InfoSeek, AltaVista and so on.

Interestingly enough, Yahoo!, the world’s best-known search engine at the time, wasn’t an Internet-wide search engine; rather, it was a large-scale searchable site directory.

The Rise of PageRank; or, How Google Took Over the World (Wide Web)

In 1998, Google founders Sergey Brin and Larry Page made history by patenting a little invention they called PageRank. The PageRank algorithm, created at Stanford University, organized web links by the link’s popularity and relevance. The result was faster and more relevant search results better tailored to searchers’ needs.

The algorithm quickly attracted the attention of leading sites including Yahoo!, who allowed Google to power the site’s search results. AdWords, Google’s advertising program, was launched in 2000. The engine’s popularity led to an entire new form of online marketing through the creation of SEO and SEM. Today, Google remains the world’s most popular search engine.

The Next Chapter of Search Engine History

Google and Bing are busy writing the next chapter of search engine history — one that features the massive rise of social search. Thanks to the introduction of Google+ and the rapid rise of Google Social Search, future search engine history books will mark 2011 and 2012 as the rise of a new era: the hallmark integration of social media and search.

Stay up-to-date with the ever-changing online landscape by signing up for our newsletter. And while you’re here, check us out on Facebook, Twitter, Tumblr or Pinterest!

About the author

admin