Comments on: Parallel web scraping in PHP: cURL multi functions

By: ina

ina — Mon, 23 Nov 2009 09:05:49 +0000

might need the resource handle $mh in the final line of close:
curl_multi_remove_handle($mh,$ch1);
(etc)

By: Torrent Search

Torrent Search — Thu, 05 Mar 2009 01:31:54 +0000

Great tutorial!

By: Eric Nagel

Eric Nagel — Tue, 03 Mar 2009 20:02:33 +0000

Wow – awesome! Just what I needed! I took your example & made it run off an array of data, using variable-variables to handle 1 to ??? instances!

Whereas 2 queries took 14 seconds before, now 11 takes 6.7 seconds.

Like Kasper said, curl_multi_remove_handle needs $mh passed to it, too.

By: Josh Fraser

Josh Fraser — Tue, 27 Jan 2009 01:17:01 +0000

Thanks for sharing this example. I made some modifications so that you can process each request as soon as it completes. It makes things a lot faster when you’re dealing with a large number of requests:

http://onlineaspect.com/2009/01/26/how-to-use-curl_multi-without-blocking/

By: Sun Location

Sun Location — Thu, 09 Oct 2008 11:54:13 +0000

Multi curl have a limit of number of calls ?

By: Kasper Garnæs

Kasper Garnæs — Tue, 29 Jul 2008 19:24:06 +0000

Great article!

It inspired me to modify my code to use multi cURL instead of cURL for a set of Google API calls.

Results from a sample of 75 calls (average of 10 runs):
- cURL: 3,7 secs
- multi cURL: 1.8 secs

A couple of minor corrections:
- curl_multi_get_content() should be curl_multi_getcontent()
- curl_multi_remove_handle($ch1) should be curl_multi_remove_handle($mh, $ch1);

By: rogue

rogue — Tue, 29 Jul 2008 13:23:30 +0000

very nice. have been thinking about doing something like this in an app i’m working on. thanks.