Scraping Links With PHP

By Justin Laing
2008-01-06


Scraping Links With PHP

In this tutorial you will learn how to build a PHP script that scrapes links from any web page.

What You’ll Learn

1. How to use cURL to get the content from a website (URL).
2. Call PHP DOM functions to parse the HTML so you can extract links.
3. Use XPath to grab links from specific parts of a page.
4. Store the scraped links in a MySQL database.
5. Put it all together into a link scraper.
6. What else you could use a scraper for.
7. Legal issues associated with scraping content.

What You Will Need

• Basic knowledge of PHP and MySQL.
• A web server running PHP 5.
• The cURL extension for PHP.
• MySQL - if you want to store the links.

Tutorial Pages:
» Scraping Links With PHP
» Get The Page Content
» Tip: Fake Your User Agent
» Using PHP’s DOM Functions To Parse The HTML
» XPath Makes Getting The Links You Want Easy
» Iterate And Store Your Links
» Your Completed Link Scraper
» What Else Could I Do With This Thing?
» Is Scraping Content Legal?


Originally posted on Makebeta

Related Tutorials: » Web Database Access from Desktop Applications » CubeCart 3.0 Installation and Configuration » PHP Site Search Made Easy » Installing and Configuring Drupal 6.1 » Desktop Application Development with PHP-GTK » Installing PHP on Windows