Web Information Retrieval

Content updated December 2011

Main text Search Engines -- Information Retrieval in Practice by W. Bruce Croft, Don Metzler, and Trevor Strohman, Addison Wesley 2010.

Other reference books include

Modern Information Retrieval, by Richardo Baeza-Yates and Berthier Ribeiro-Neto. (site at Brazil)
A U.S. mirror site at www.sims.berkeley.edu/~hearst/irbook/.
An Introduction to Information Retrieval by Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schutze, August 2006.
Information Retrieval, by Van Rijsbergen, available on-line www.dcs.gla.ac.uk/Keith/Preface.html.
Automatic Text Processing by Gerard Salton, Addison-Wesley, 1989.
Finding Out About: A Cognitive Perspective on Search Engine Technology and the WWW, by Richard K. Below, Cambridge University Press, 2001.
Internet Agents: Spiders, Wanders, Brokers, and Bots, by Fah-Chun Cheong, New Riders Publishers, 1996.
Data Mining: Concepts and Techniques, by Jiawei Han and Micheline Kamber, Morgan Kaufmann, 2001.
Data Mining Methods for Knowledge Discovery, by Krzysztof Cios, Witold Pedrycz, Roman Swiniarski, Kluwer, 1998.
Information Storage and Retrieval, by Robert R. Korfhage, John Wiley & Sons, 1997

A Java server example
A C server and client example
A Java client example
Some implemenations of Porter's algorithm
Example of extracting URLs from a web page
Example of file access in Java.
Traverse a directory tree in C/C++ and in Java.
A simple class to check robots.txt at a given website.
A utility class to manipulate string URLs, e.g. convert relative path to absolute path.
Binary search for insertion
A simple example and some instructions on mysql on Linux and Solaris.

The Google architecture paper, Sergey Brin and Lawrence Page, "The Anatomy of a Large-Scale Hypertextual Web Search Engine", 7th IWWW Conference, Brisbane, Australia, 14-18 April 1998.
David Gibson, Jon Kleinberg, Prabhakar Raghavan Inferring Web Communities from Link Topology, Proceedings of the 9th ACM Conference on Hypertext and Hypermedia, 1998.
Mei Kobayashi and Koichi Takeda, "Information Retrieval on the Web", ACM Computing Surveys, 32(2), pp. 144-173, 2000.
Raymond Kosala and Hendrik Blockeel, "Web Mining Research: A survey", SIGKDD Explorations, July 2000, 2(1), pp. 1-15.
Z. Wu, W. Meng, C. Yu, and Z. Li, "Towards a highly-scalable and effective metasearch engine", WWW10, 2001.
Sergey Melnik, Sriram Raghavan, Beverly Yang, Hector Garcia-Monina, "Building a distributed full-text index for the Web", WWW10, 2001.
Soumen Chakrabarti, Byron E. Dom, David Gibson, Jon Kleinbeig, Ravi Kumar, Prabhakar Raghavan, Sridhar Rajagopalan and Andrew Tomkins, "Mining the Link Structure of the World Wide Web", WWW10, 2001.
R. Srikant and Y. Yang, "Mining Web log to improve Website organization", WWW10, 2001.
L. Finkelstein, E. Gabrilovich, Y. Matias, E. Rivlin, Z. Solan, G. Wolfman, and E. Ruppin, "Placing search in context: The concept revisited".
D. Haines and W. Bruce Croft, "Relevance Feedback and Inference Networks",