In response to Wil’s comment regarding the PHP framework performance comparisons I made in my previous post, I have decided to post the results I got.
My decision not post them initially was due to the benchmarks not being done in complete isolation (a seperate client and server machine) but the scores relative to each should still be accurate and that’s what I am testing for.

Furthermore, while I only used requests/sec as a performance yardstick, I do realize that other factors do affect a web application’s response in the real world.

The Tools

The tests were run on a 1.8Ghz AMD sempron computer with 512Mb RAM running Ubuntu Gutsy (7.10) Desktop.
I used httperf to run the tests although I did run some of them again using apachebench and the results were consistent.

I created the same front page with CakePHP, CodeIgniter, Zend Framework and Ruby on Rails. I also duplicated the same functionality in HTML and in PHP using procedural code (aka. spaghetti code) to act as baselines.

I decided to include data access in the tests and the tests involve the application fetching four rows from the database and rendering the results.

View the sample page.

The test files are also available for download for those interested in them. The sql file for the mysql database is also included. The code assumes a database root user with no password.

• Get the tar file
• Get the zip file

The Results

All scores are requests per second. Higher is better.

• No PHP code cache
  All frameworks used an ORM (of sorts, in the case of codeigniter). The Zend Framework used Zend_DB_Table and CodeIgniter used ActiveRecord.

  	Run 1	Run 2	Run 3	Run 4	Average
  Baseline HTML	1327.5	1326.5	1328.6	1329.1	1327.9
  Baseline PHP	331.6	332.1	331.4	332.0	331.8
  CakePHP	3.6	3.7	3.8	3.5	3.7
  CodeIgniter	21.5	21.2	21.7	21.7	21.5
  Zend Framework	9.3	9.1	9.2	9.3	9.2

• With eAccelerator PHP code cache
  All frameworks used an ORM (of sorts, in the case of CodeIgniter). The Zend Framework used Zend_DB_Table and CodeIgniter used ActiveRecord.

  	Run 1	Run 2	Run 3	Run 4	Average
  CakePHP	8.0	8.0	8.0	7.2	7.8
  CodeIgniter	98.2	98.1	98.3	98.3	98.2
  Zend Framework	33.2	33.3	33.5	33.6	33.4

• With APC PHP code cache
  All frameworks used an ORM (of sorts, in the case of CodeIgniter). The Zend Framework used Zend_DB_Table and CodeIgniter used ActiveRecord.

  	Run 1	Run 2	Run 3	Run 4	Average
  CakePHP	7.3	7.3	7.3	7.3	7.3
  CodeIgniter	97.5	98.0	96.6	98.3	97.6
  Zend Framework	32.8	33.3	31.8	32.7	32.7

• With APC PHP code cache, No ORM
  The Zend Framework used Zend_DB and I disabled ActiveRecord in CodeIgniter. I couldn’t figure out how to disable the ORM in cakePHP so I left it out.

  	Run 1	Run 2	Run 3	Run 4	Average
  CodeIgniter	106.4	105.6	106.3	106.6	106.2
  Zend Framework	42.6	42.8	42.9	43.0	42.8

• With APC PHP code cache, No database calls
  This tests the impact of the database call on the overall performance. I sent an empty result set to the view directly from the controller, thus bypassing the model.

  	Run 1	Run 2	Run 3	Run 4	Average
  CodeIgniter	118.1	118.3	117.3	118.2	118.0
  Zend Framework	51.9	52.1	52.1	52.2	52.0

• Ruby on Rails comparison
  I added this test to see how the PHP frameworks stcked up against Ruby on Rails. I also used the chance to try out Passenger (mod_rails) and Ruby Enterprise.

  	Run 1	Run 2	Run 3	Run 4	Average
  ROR with 1 Mongrel	88.1	85.1	84.9	84.8	85.7
  ROR with Passenger	85.2	97.3	86.3	84.2	88.2
  ROR with Passenger and Ruby Enterprise	89.0	99.2	98.9	98.6	96.4

Conclusion

CodeIgniter is over twice the speed of the Zend framework in all cases and CakePHP is a lot slower than the other two PHP frameworks. I do admit that considering my experience with CodeIgniter, I might have inadvertently set it up optimally without doing the same to the other two frameworks. I have posted the files and I welcome comments from “the experts”.

Update

We have decided to use the Zend framework so obviously outright performance is not the only factor in the choice of framework. The results we are currently getting are fast enough for us and in our existing application, the database is the bottleneck not the PHP code.

I have also done a few more tests on a production-grade dual-core server with 2Gb of RAM running Centos 5. This time, all benchmarks were from a seperate client computer on the same network. I used the exact same files as the previous results except for CakePHP where I used the recently released RC2. The results follow.

• No PHP code cache

  All frameworks used an ORM (of sorts, in the case of codeigniter). The Zend Framework used Zend_DB_Table and CodeIgniter used ActiveRecord.

  	Run 1	Run 2	Run 3	Run 4	Average
  Baseline HTML	3431.2	3311.8	3427.7	3395.0	3391.4
  Baseline PHP	1912.1	1932.3	1983.3	1911.3	1934.7
  CakePHP	15.6	15.6	15.6	15.6	15.6
  CodeIgniter	83.5	83.0	82.0	83.2	82.9
  Zend Framework	34.7	34.6	34.6	34.6	34.6

• With eAccelerator PHP code cache

  All frameworks used an ORM (of sorts, in the case of CodeIgniter). The Zend Framework used Zend_DB_Table and CodeIgniter used ActiveRecord.

  	Run 1	Run 2	Run 3	Run 4	Average
  CakePHP	36.0	36.1	36.1	36.2	36.1
  CodeIgniter	383.3	377.9	371.8	385.2	379.5
  Zend Framework	129.2	128.5	129.0	128.9	128.9

• With the Zend Platform

  All frameworks used an ORM (of sorts, in the case of CodeIgniter). The Zend Framework used Zend_DB_Table and CodeIgniter used ActiveRecord.

  	Run 1	Run 2	Run 3	Run 4	Average
  CakePHP	24.7	24.8	24.9	24.8	24.8
  CodeIgniter	255.0	256.6	254.1	253.8	254.9
  Zend Framework	83.5	84.4	83.8	83.4	83.8

We are about to start a project from scratch at my new job and have been evaluating PHP frameworks. We’ve shortlisted CakePHP, CodeIgniter, Symfony and Zend.
I have put them through their paces by building the same application with all four of them (a simple wiki application) and hopefully, we’ll settle on one soon enough.

Full Disclosure: I have tried to be as unbiased as I possibly can but I’m already a CodeIgniter fan. That said, the company I work for is a Zend Partner (we already use the Zend Platform and Zend Studio) and I can’t help factoring that in.

Although the initial plan was to review four PHP frameworks, this post has become a direct CodeIgniter to Zend Framework comparison. I have had to exclude Symfony and CakePHP from the list after spending a few hours going through all four frameworks for the following reasons:

• Learning curve:
  Both symphony and CakePHP have a very steep learning curve. CakePHP has strict rules about database table names, where files should be placed, method names and class names. Symfony stores its configuration in .yml format (requires learning although it’s not really that hard) and a lot of the interaction with the application is through a console. Creating database tables, data models and various other files are done using the command line.
• Strict ORM:
  CakePHP and Symphony have full-blown object-relational mappers (ORM) to provide access to the database and these cannot be disabled without a lot of effort. These ORM have strict rules and conventions which must be adhered to for the application to work.
  In contrast the Zend Framework and CodeIgniter are flexible about using models and how they are used. Using a model is optional and while they each have data mappers, applications can work without them. The application will be extremely database intensive and we would rather not be limited in our choices.
• Flexibility:
  The Zend Framework and CodeIgniter are more flexible than the other two frameworks.

The Duel

Summary

I was rather surprised at the performance difference (measured using apachebench loading the home page with one call to the database to retrieve four rows). I expected the efficiency of using PHP5 only features to make up for the extra size of the Zend framework.

The Zend Framework advantages include:

• The “official PHP framework”.
• My workplace is already a Zend “partner”.
• Full-featured layout and template system.
• Massive number of classes and components.
• Extremely flexible.
• More advanced database library.
• More advanced validation library.
• Internationalization support.

CodeIgniter advantages include:

• Extremely easy to setup.
• Lower learning curve then the Zend Framework.
• More accessible documentation.
• Concise syntax – The Zend Framework syntax is wordier.
• 100% faster than the Zend framework.

There isn’t any clear cut “winner” here in my opinion and we still haven’t chosen one yet.

Update: the benchmarks are now available

I use Windows XP as my development platform and this is more of a note from me to me so I don’t have to Google for it next time.

I also use WampServer for Apache, MySQL and PHP and while I’ve only tested on that stack, I believe it to be generic enough.

• Open the Apache httpd.conf file and uncomment the line ‘LoadModule rewrite_module modules/mod_rewrite.so’ (remove the pound ‘#’ sign from in front of the line).
• If the lines ‘ClearModuleList’ and ‘AddModule mod_rewrite.c’ are in your httpd.conf file, make sure they are not commented out.
• Find the section with Document Root: DocumentRoot “C:/path/to/my/root”
• Modify the option for AllowOverride to look like this:
  <Directory />
      Options Indexes FollowSymLinks
      AllowOverride All
  </Directory>
• Restart Apache

Apache mod-rewrite should now be working.