Over on the AVNet Labs blog, Ekerete Akpan has posted a few results from his recent framework benchmarks to much controversy. His tests covered a few major PHP frameworks, such as CodeIgniter, CakePHP and Zend Framework, along with baseline results for serving static HTML and procedural PHP versions of the same page. He also covered some basic Ruby on Rails, including Passenger (mod_rails?) and Ruby Enterprise. Here are some of his more interesting findings.
After running the tests on a production grade box, he recorded these basic results, without any caching in place.
|
Run 1 |
Run 2 |
Run 3 |
Run 4 |
Average |
| Baseline HTML |
3431.2 |
3311.8 |
3427.7 |
3395.0 |
3391.4 |
| Baseline PHP |
1912.1 |
1932.3 |
1983.3 |
1911.3 |
1934.7 |
| CakePHP |
15.6 |
15.6 |
15.6 |
15.6 |
15.6 |
| CodeIgniter |
83.5 |
83.0 |
82.0 |
83.2 |
82.9 |
| Zend Framework |
34.7 |
34.6 |
34.6 |
34.6 |
34.6 |
Some basic indications are obvious - frameworks create massive overheads, with hefty overall performance costs. Per these results, using CakePHP over a static HTML site was 226 times slower. CodeIgniter fared the best, at about 40 times slower, while the Zend Framework was somewhere in between.
While the actual numbers may be surprising, how each framework fared in relation to the others is not.
All of these frameworks provide levels of abstraction, taking care of many common tasks and moving the logic away from the developer’s application. Each framework offers a varying level of this “hand holding”, and it’s generally accepted that CakePHP leads the race in this area. By making more assumptions and relying on more default behaviours, Cake will always have a far greater overhead.
CodeIgniter, on the other hand, is far more efficient in what it does - but doesn’t do as much. An application built on CodeIgniter will always be more efficient than one built in Cake, but development time will be considerably longer.
Another interesting area to note was ORM - Cake’s inbuilt ORM layer couldn’t be reasonably “disabled” to run benchmarks without the added overhead, and so the benchmark of frameworks without their ORM layer excluded Cake entirely. Ekerete also notes that CodeIgniter doesn’t have much of an ORM implementation, possibly leading to its vastly increased performance over the other frameworks.
However, the final point to consider was caching. The performance (measured in requests per second) of CodeIgniter and Zend Framework increased by about 4x when the eAccelerator cache was used. On the other hand, Cake’s requests per second figure barely doubled, suggesting that Cake’s architecture imposes some limitations on the performance gains achievable through caching. With eAccelerator, CodeIgniter was serving over ten times as many requests per second than Cake.
In the end, the choice is really up to the developer. If your applications tend to grow in complexity very quickly, your choice of framework (or language, for that matter) will be increasingly irrelevant. On the other hand, if you’re building small applications, CakePHP will clearly save you time in development, CodeIgniter will offer massive performance benefits, and Zend will give you a reasonable middle ground.
Check out the post on AVNet Labs for more details.
Akash Mehta in PHP |
on Thursday, July 3rd, 2008 at 7:00 am.
RSS 2.0 |
Leave a response | Trackback
