Dries recently published Drupal 5 performance benchmark results.

I would like to share some of my thoughts about the subject.

First, of course, we should appreciate that Dries took time and performed the test. Better some results than none. However, it is only natural that objective audience mostly meets benchmark results with certain level of skepticism. I won’t be an exception. Personally, I especially do not like or trust black-box performance tests. If you tell me that A is faster than B, tell me why, show me the specific part of A that makes it faster. It will make results much more interesting, trustworthy and educational.

We’ve heard about the CSS optimization in Drupal 5 via consolidation of many CSS files into one. We were told that the same technique is on its way for javascripts as well. It’s great, it really is, but honestly I think such optimization mostly affects scalability, not necessarily pure performance at some arbitrary level. Which brings us to the next point: I don’t like performance tests done on one arbitrary level of load. A serious performance benchmark should show me the performance at several load levels, so I can at least try to see the scalability trend.

Having criticized the methodology, let’s look at the results, per se. What do they say?

As we can see, the performance without caching is comparable if not equal. The difference is only visible with caching turned on. Apparently Drupal 5 has better tuned or more aggresive caching. I am not sure how I feel about it, honestly. Generally, I do not like systems with stale cache. I firmly believe that systems MUST be tuned to the extreme with all non-stale mechanisms (cache that does not cause stale information is fine because it is smart cache). Only once that is done, can we allow extreme caching for cases like “getting slashdotted” etc.

To conclude, I think Drupal is CMS with the best architecture and set of modules, there is today. It has some distance to go for performance and being pragmatic rather than falsely cheerful will be more helpful in that journey.

There was some extraneous cheer in the comments to Dries article but Dries himself gets it: “Furthermore, [test results] suggest that for Drupal 6, we need to look at improving the page generation time of non-cached pages. Let’s make that an action item”. I have no doubt that Drupal will get much better. I myself intend to contribute to that effort, too. It’s not like I just sit here and criticize :) I love Drupal!