Make Database Queries Without the Database
By Brian Goetz2005-07-14
I recently advised on a project that involved a fair bit of Web crawling. As the program crawled the various Web sites, it would build a database of the sites and pages crawled, the links each page contained, the results of analysis on each pages, and so on. The end result was to be a set of reports detailing which sites and pages were crawled, which were followed, which links were broken, which pages had errors, page metrics that were calculated, and so on. At the outset, no one knew exactly what reports were going to be needed or what format they should take -- just that there would need to be reports. This suggested that the report development phase was going to be iterative, with several rounds of feedback, tweaking, and possibly starting over with a different organization. The only specified reporting requirement was that the reports should be presented as XML, or possibly HTML. As a result, the process of developing and modifying the reports would have to be lightweight, because the reporting requirements were going to be "discovered dynamically," rather than specified up-front.
Tutorial pages:
|
First published by IBM developerWorks
|
|||||||||
You might also want to check these out:
|
Link to This Tutorial Page!
