Make Database Queries Without the Database

By Brian Goetz
2005-07-14


Borrowing a data model can simplify development and enhance performance

I recently advised on a project that involved a fair bit of Web crawling. As the program crawled the various Web sites, it would build a database of the sites and pages crawled, the links each page contained, the results of analysis on each pages, and so on. The end result was to be a set of reports detailing which sites and pages were crawled, which were followed, which links were broken, which pages had errors, page metrics that were calculated, and so on. At the outset, no one knew exactly what reports were going to be needed or what format they should take -- just that there would need to be reports. This suggested that the report development phase was going to be iterative, with several rounds of feedback, tweaking, and possibly starting over with a different organization. The only specified reporting requirement was that the reports should be presented as XML, or possibly HTML. As a result, the process of developing and modifying the reports would have to be lightweight, because the reporting requirements were going to be "discovered dynamically," rather than specified up-front.

Tutorial Pages:
» Borrowing a data model can simplify development and enhance performance
» No database needed
» My kingdom for a data model
» XQuery to the rescue
» Summary
» Resources


First published by IBM developerWorks

Related Tutorials: » All about JAXP, Part 1 » Load List Values for Improved Efficiency » 2 Ways To Implement Session Tracking » A Simple Way to Read an XML File in Java » Develop Aspect-Oriented Java Applications with Eclipse and AJDT » Java Validation With Dynamic Proxies