Maximize your Shallow Web first

Walk first, Run Second

There are many search engines, but they all do the same thing. They crawl the web and index what they find. There are a few big ones, like Google, Yahoo, Bing, and Dogpile.

There are many more smaller boutique search engines that specialize, like Pipl, Zuula, and others. Each has a strength, these should be found and added to your Source Book.

Almost ever search engine uses some form of boolean logic to help you narrow your search:

  • If you wanted to find just keywords on a single site on Google, say the word ‘books’ on treasurenet.com use [books site:treasurenet.com]
  • If you wanted keywords in a specific order ie “Black and Tan” when looking for beer use quotes, like [“black and tan” beer]
  • If you don’t want certain words in your results use a minus sign, no space, and the word. ie, we don’t want results for tanning salons so we use [“black and tan” beer -tanning]

Jog over to the Kind-of-Deep Web

Google has some nice deep web features, namely a book search (book.google.com) for search current and out-of-print books and an academic search (scholar.google.com) for scientific papers.

I call it Kind-of-Deep Web, as it has been reached and traversed by a bot. The bot has found that there is something there but it doesn’t know what it is exactly, but it thinks it know something from the meta data. An example would be a bot that finds an image of a 1715 ships manifest. The meta data told it was a ships manifest from 1715, but it doesn’t know what is on the manifest.

Some search tools I can recommend would be:

  1. Zuula.com
  2. scholar.google.com
  3. dogpile.com
  4. Deeperweb.com – Searches for best hits as a blog, news, web general, research search, etc.
  5. Touchgraph