June 30th, 2008 | Ekerete
In response to Wil’s comment regarding the PHP framework performance comparisons I made in my previous post, I have decided to post the results I got.
My decision not post them initially was due to the benchmarks not being done in complete isolation (a seperate client and server machine) but the scores relative to each should still be accurate and that’s what I am testing for.
Furthermore, while I only used requests/sec as a performance yardstick, I do realize that other factors do affect a web application’s response in the real world.
The Tools
The tests were run on a 1.8Ghz AMD sempron computer with 512Mb RAM running Ubuntu Gutsy (7.10) Desktop.
I used httperf to run the tests although I did run some of them again using apachebench and the results were consistent.
I created the same front page with CakePHP, CodeIgniter, Zend Framework and Ruby on Rails. I also duplicated the same functionality in HTML and in PHP using procedural code (aka. spaghetti code) to act as baselines.
I decided to include data access in the tests and the tests involve the application fetching four rows from the database and rendering the results.
View the sample page.
The test files are also available for download for those interested in them. The sql file for the mysql database is also included. The code assumes a database root user with no password.
The Results
All scores are requests per second. Higher is better.
- No PHP code cache
All frameworks used an ORM (of sorts, in the case of codeigniter). The Zend Framework used Zend_DB_Table and CodeIgniter used ActiveRecord.
|
Run 1
|
Run 2
|
Run 3
|
Run 4
|
Average
|
| Baseline HTML |
1327.5
|
1326.5
|
1328.6
|
1329.1
|
1327.9
|
| Baseline PHP |
331.6
|
332.1
|
331.4
|
332.0
|
331.8
|
| CakePHP |
3.6
|
3.7
|
3.8
|
3.5
|
3.7
|
| CodeIgniter |
21.5
|
21.2
|
21.7
|
21.7
|
21.5
|
| Zend Framework |
9.3
|
9.1
|
9.2
|
9.3
|
9.2
|
- With eAccelerator PHP code cache
All frameworks used an ORM (of sorts, in the case of CodeIgniter). The Zend Framework used Zend_DB_Table and CodeIgniter used ActiveRecord.
|
Run 1
|
Run 2
|
Run 3
|
Run 4
|
Average
|
| CakePHP |
8.0
|
8.0
|
8.0
|
7.2
|
7.8
|
| CodeIgniter |
98.2
|
98.1
|
98.3
|
98.3
|
98.2
|
| Zend Framework |
33.2
|
33.3
|
33.5
|
33.6
|
33.4
|
- With APC PHP code cache
All frameworks used an ORM (of sorts, in the case of CodeIgniter). The Zend Framework used Zend_DB_Table and CodeIgniter used ActiveRecord.
|
Run 1
|
Run 2
|
Run 3
|
Run 4
|
Average
|
| CakePHP |
7.3
|
7.3
|
7.3
|
7.3
|
7.3
|
| CodeIgniter |
97.5
|
98.0
|
96.6
|
98.3
|
97.6
|
| Zend Framework |
32.8
|
33.3
|
31.8
|
32.7
|
32.7
|
- With APC PHP code cache, No ORM
The Zend Framework used Zend_DB and I disabled ActiveRecord in CodeIgniter. I couldn’t figure out how to disable the ORM in cakePHP so I left it out.
|
Run 1
|
Run 2
|
Run 3
|
Run 4
|
Average
|
| CodeIgniter |
106.4
|
105.6
|
106.3
|
106.6
|
106.2
|
| Zend Framework |
42.6
|
42.8
|
42.9
|
43.0
|
42.8
|
- With APC PHP code cache, No database calls
This tests the impact of the database call on the overall performance. I sent an empty result set to the view directly from the controller, thus bypassing the model.
|
Run 1
|
Run 2
|
Run 3
|
Run 4
|
Average
|
| CodeIgniter |
118.1
|
118.3
|
117.3
|
118.2
|
118.0
|
| Zend Framework |
51.9
|
52.1
|
52.1
|
52.2
|
52.0
|
- Ruby on Rails comparison
I added this test to see how the PHP frameworks stcked up against Ruby on Rails. I also used the chance to try out Passenger (mod_rails) and Ruby Enterprise.
|
Run 1
|
Run 2
|
Run 3
|
Run 4
|
Average
|
| ROR with 1 Mongrel |
88.1
|
85.1
|
84.9
|
84.8
|
85.7
|
| ROR with Passenger |
85.2
|
97.3
|
86.3
|
84.2
|
88.2
|
| ROR with Passenger and Ruby Enterprise |
89.0
|
99.2
|
98.9
|
98.6
|
96.4
|
Conclusion
CodeIgniter is over twice the speed of the Zend framework in all cases and CakePHP is a lot slower than the other two PHP frameworks. I do admit that considering my experience with CodeIgniter, I might have inadvertently set it up optimally without doing the same to the other two frameworks. I have posted the files and I welcome comments from “the experts”.
Update
We have decided to use the Zend framework so obviously outright performance is not the only factor in the choice of framework. The results we are currently getting are fast enough for us and in our existing application, the database is the bottleneck not the PHP code.
I have also done a few more tests on a production-grade dual-core server with 2Gb of RAM running Centos 5. This time, all benchmarks were from a seperate client computer on the same network. I used the exact same files as the previous results except for CakePHP where I used the recently released RC2. The results follow.
- No PHP code cache
All frameworks used an ORM (of sorts, in the case of codeigniter). The Zend Framework used Zend_DB_Table and CodeIgniter used ActiveRecord.
|
Run 1
|
Run 2
|
Run 3
|
Run 4
|
Average
|
| Baseline HTML |
3431.2
|
3311.8
|
3427.7
|
3395.0
|
3391.4
|
| Baseline PHP |
1912.1
|
1932.3
|
1983.3
|
1911.3
|
1934.7
|
| CakePHP |
15.6
|
15.6
|
15.6
|
15.6
|
15.6
|
| CodeIgniter |
83.5
|
83.0
|
82.0
|
83.2
|
82.9
|
| Zend Framework |
34.7
|
34.6
|
34.6
|
34.6
|
34.6
|
- With eAccelerator PHP code cache
All frameworks used an ORM (of sorts, in the case of CodeIgniter). The Zend Framework used Zend_DB_Table and CodeIgniter used ActiveRecord.
|
Run 1
|
Run 2
|
Run 3
|
Run 4
|
Average
|
| CakePHP |
36.0
|
36.1
|
36.1
|
36.2
|
36.1
|
| CodeIgniter |
383.3
|
377.9
|
371.8
|
385.2
|
379.5
|
| Zend Framework |
129.2
|
128.5
|
129.0
|
128.9
|
128.9
|
- With the Zend Platform
All frameworks used an ORM (of sorts, in the case of CodeIgniter). The Zend Framework used Zend_DB_Table and CodeIgniter used ActiveRecord.
|
Run 1
|
Run 2
|
Run 3
|
Run 4
|
Average
|
| CakePHP |
24.7
|
24.8
|
24.9
|
24.8
|
24.8
|
| CodeIgniter |
255.0
|
256.6
|
254.1
|
253.8
|
254.9
|
| Zend Framework |
83.5
|
84.4
|
83.8
|
83.4
|
83.8
|
Posted in PHP | 55 Comments »
June 24th, 2008 | Ekerete
We are about to start a project from scratch at my new job and have been evaluating PHP frameworks. We’ve shortlisted CakePHP, CodeIgniter, Symfony and Zend.
I have put them through their paces by building the same application with all four of them (a simple wiki application) and hopefully, we’ll settle on one soon enough.
Full Disclosure: I have tried to be as unbiased as I possibly can but I’m already a CodeIgniter fan. That said, the company I work for is a Zend Partner (we already use the Zend Platform and Zend Studio) and I can’t help factoring that in.
Although the initial plan was to review four PHP frameworks, this post has become a direct CodeIgniter to Zend Framework comparison. I have had to exclude Symfony and CakePHP from the list after spending a few hours going through all four frameworks for the following reasons:
- Learning curve:
Both symphony and CakePHP have a very steep learning curve. CakePHP has strict rules about database table names, where files should be placed, method names and class names. Symfony stores its configuration in .yml format (requires learning although it’s not really that hard) and a lot of the interaction with the application is through a console. Creating database tables, data models and various other files are done using the command line.
- Strict ORM:
CakePHP and Symphony have full-blown object-relational mappers (ORM) to provide access to the database and these cannot be disabled without a lot of effort. These ORM have strict rules and conventions which must be adhered to for the application to work.
In contrast the Zend Framework and CodeIgniter are flexible about using models and how they are used. Using a model is optional and while they each have data mappers, applications can work without them. The application will be extremely database intensive and we would rather not be limited in our choices.
- Flexibility:
The Zend Framework and CodeIgniter are more flexible than the other two frameworks.
The Duel
| |
CodeIgniter |
Zend Framework |
| Set Up |
CodeIgniter is very easy to set up. Copy all the framework files to the web server and it’s good to go. It also has a small folder size – about 2.1 Mb and I could display the default home page less than five minutes after I started the set-up. |
The Zend Framework requires a bit of effort to setup the project. It requires the creation of a bootstrap file with all the initialisation stuff it. The framework is relatively large – about 12.4Mb and the set-up process took about 19 minutes. |
| Documentation |
The documentation is very well-structured and organized although it is a bit less detailed than the Zend framework documentation.
CodeIgniter also has forums and a wiki which feature a lot of user-submitted code. |
The Zend Framework has very detailed documentation with a lot of examples. It is less organised than the CodeIgniter docs in my view although this could be down to the afore-mentioned detail and the large number of components available in the framework.ZF also has a wiki with a few tutorials. |
| Templating |
CodeIgniter includes a template parser class although in my opinion its use is limited as it does not support logic (e.g. if statements) in the views.However the CI recommendation is to use PHP tags in the views. |
The Zend framework includes a Layout class designed to provide a common layout (or multiple layouts) for the entire website or application. It uses PHP tags for templating although it does provide an abstract view class which can be extended with a 3rd party template library. |
| Components |
CI has a lot of libraries and helpers to simplify the developer’s life.While it does have less of these than ZF, in the main, the usage of the CI variants is simpler. |
ZF has a massive number of classes and components.These are well documented although the usage is usually a bit more difficult than in CI. |
| Database Access |
CI includes a database class which handles the database connection. The database class can be used for standard SQL queries creating, retrieving, updating and deleting data in the standard PHP way.CI also includes an active record class which is a modified version of the Active Record Database Pattern. This pattern allows information to be retrieved, inserted, and updated in your database with minimal scripting. In some cases only one or two lines of code are necessary to perform a database action.Beyond simplicity, a major benefit to using the Active Record features is that it allows the creation of database independent applications, since the query syntax is generated by each database adapter. It also allows for safer queries, since the values are escaped automatically by the system. |
Zend_Db and its related classes provide a simple SQL database interface for Zend Framework. It allows for standard SQL queries but simplifies retrieving the SQL results.It also includes an ORM using both the Table Data Gateway and the Row Data Gateway. These represent the database table and row respectively as objects and can drastically reduce development speed.The downside is a slight performance deficit when compared to the modified active record pattern used in CodeIgniter which does not use objects as extensively.Zend_Db can also model table relationships in PHP classes making database joins a breeze. |
| Flexibility |
CI is very flexible allowing almost all defaults to be modified. |
ZF is simply a collection classes and as such any file or folder can be placed anywhere as long as the location is added to the bootstrap file. |
| Validation |
Data validation in CodeIgniter is handled via a validation class. A set of rules gets defined and assigned to the validation object.The validation object automatically validates the data passed via the URL or form. From there, the programmer can decide how that gets handled.The validation class can also help automate some of the process of setting error messages for specific fields. |
The Zend_Validate component provides a set of commonly needed validators. It also provides a simple validator chaining mechanism by which multiple validators may be applied to a single datum in a user-defined order.In ZF, each validator is a separate class and the class is added to the data (like a filter) rather than the data being passed into the class like it is in CodeIgniter. |
| Forms |
The Form Helper file in CI contains functions that assist in working with forms.It aids in the generation of form fields although it does not completely eliminate the need to write HTML code. |
Zend_Form simplifies form creation and handling. It handles element filtering and validation, escaping data and form rendering.Using Zend_Form, ZF can represent a form completely in PHP code including labels, validation and error messages. |
| Performance |
CI has about double the performance of the Zend Framework. |
The Zend Framework is about half as fast as CodeIgniter. |
| Testing |
CodeIgniter has a unit testing class but it encourages mixing the test code with the actual source code so I don’t recommend it.A third-party extension for SimpleTest is available though.Using PHPUnit with the CI classes should also be possible. |
The Zend Framework does not have a built-in unit testing class but the core classes use PHPUnit as their test framework and this can be extended to include any additional classes.Using SimpleTest with the ZF classes should also be possible. |
| Internationalisation |
No |
Yes |
| License |
BSD-style |
New BSD |
Summary
I was rather surprised at the performance difference (measured using apachebench loading the home page with one call to the database to retrieve four rows). I expected the efficiency of using PHP5 only features to make up for the extra size of the Zend framework.
The Zend Framework advantages include:
- The “official PHP framework”.
- My workplace is already a Zend “partner”.
- Full-featured layout and template system.
- Massive number of classes and components.
- Extremely flexible.
- More advanced database library.
- More advanced validation library.
- Internationalization support.
CodeIgniter advantages include:
- Extremely easy to setup.
- Lower learning curve then the Zend Framework.
- More accessible documentation.
- Concise syntax – The Zend Framework syntax is wordier.
- 100% faster than the Zend framework.
There isn’t any clear cut “winner” here in my opinion and we still haven’t chosen one yet.
Update: the benchmarks are now available
Posted in PHP | 5 Comments »
June 13th, 2008 | Ekerete
I use Windows XP as my development platform and this is more of a note from me to me so I don’t have to Google for it next time.
I also use WampServer for Apache, MySQL and PHP and while I’ve only tested on that stack, I believe it to be generic enough.
- Open the Apache httpd.conf file and uncomment the line ‘LoadModule rewrite_module modules/mod_rewrite.so’ (remove the pound ‘#’ sign from in front of the line).
- If the lines ‘ClearModuleList’ and ‘AddModule mod_rewrite.c’ are in your httpd.conf file, make sure they are not commented out.
- Find the section with Document Root: DocumentRoot “C:/path/to/my/root”
- Modify the option for AllowOverride to look like this:
<Directory />
Options Indexes FollowSymLinks
AllowOverride All
</Directory>
- Restart Apache
Apache mod-rewrite should now be working.
Posted in PHP | No Comments »
May 20th, 2008 | Ekerete
While trying to streamline my growing book collection, I thought to do a post on the books that have been most useful. So here goes:
- The Rails Way by Obie Fernandez
Useful both as a reference book and a source of advice from the trenches. Covers Rails 2. Smashing.
- The ZEN of CSS Design by Dave Shea, Molly E. Holzschlag
The definitive CSS showcase.
- Prototype and Scriptaculous in Action by Dave Crane, Bear Bibeault, Tom Locke
Work with Prototype and Scriptaculous? You need this.
- jQuery in Action by Bear Bibeault, Yehuda Katz
As above but for the jQuery javascript library.
- DOM Scripting by Jeremy Keith
Absolutely recommended as an intro to working with the Document Object Model.
- Bulletproof Web Design by Dan Cederholm
Lots of useful CSS tips and tricks.
- Bulletproof Ajax by Jeremy Keith
Very friendly in-depth intro to ajax. Useful even if you use a framework.
- RESTful Web Services by Leonard Richardson, Sam Ruby
The title says it all. Web Services the modern way.
- Rails Cookbook by Rob Orsini
If you need to choose just one Rails recipe book, choose this.
- Don’t Make Me Think! by Steve Krug
Engrossing usability and user interface design book. Read it from cover to cover in a day.
- PHP 5 Objects, Patterns, and Practice by Matt Zandstra
All you need to get started with Object Oriented Programming with PHP 5.
- Agile Web Development with Rails by Dave Thomas, David Hansson, Leon Breedt, Mike Clark, James Davidson, Justin Gehtland, Andreas Schwarz
The “official” rails book. Has everything you need to get started with Rails.
- Rules of Management by Richard Templar
Lots of useful advice for managing resources.
- Php|architect’s Guide to Programming with Zend Framework by Cal Evans
Currently the only Zend Framework book on sale and it’s really good.
- No Nonsense XML Web Development With PHP by Thomas Myer
Very good. Walks through an article management project while introducing different aspects of XML with PHP.
View the full Image
Posted in General | No Comments »
