Maximize your Shallow Web first
Almost ever search engine uses some form of boolean logic to help you narrow your search:
- If you wanted to find just keywords on a single site on Google, say the word ‘books’ on treasurenet.com use [books site:treasurenet.com]
- If you wanted keywords in a specific order ie “Black and Tan” when looking for beer use quotes, like ["black and tan" beer]
- If you don’t want certain words in your results use a minus sign, no space, and the word. ie, we don’t want results for tanning salons so we use ["black and tan" beer -tanning]
Jog over to the Kind-of-Deep Web
I call it Kind-of-Deep Web, as it has been reached and traversed by a bot. The bot has found that there is something there but it doesn’t know what it is exactly, but it thinks it know something from the meta data. An example would be a bot that finds an image of a 1715 ships manifest. The meta data told it was a ships manifest from 1715, but it doesn’t know what is on the manifest.
Some search tools I can recommend would be:
- Deeperweb.com – Searches for best hits as a blog, news, web general, research search, etc.
- Touchgraph – a wickedly nifty clustering tool that may lead you to new sources
[ Editor - expand out to improve google searches -mt 22MAR12 ]