WordPress Release Stats

Before we get to test results, first thing is to see how this 4 version compare when it comes to size, number of files, hooks, requirements and few other things. Latest version is the version we used to test here.

Test Environment

For tests I have used WordPress 3.0.4, WordPress 3.1.4, WordPress 3.2.1 and WordPress 3.3. Test machine is CentOS powered VPS server with PHP 5.3.3 and mySQL 5.5.14, with PHP memory limit set to 256MB. PHP runs with eAccelerator enabled. All WP installations used only GD Press Tools 4.3.1 Pro plugin, and all had exactly the same content. GD Press Tools Pro had all its security related features activated, and they add about 3 SQL queries per page, and these queries are counted in the results below. Since I run GD Press Tools Pro as a part of all my WordPress installations, I think of it as logical and essential expansion of WordPress.

Each test for each WP version is repeated 10 times, highest and lowest results are removed, and the average is calculated from the rest of the measured values. Same method is used for server and client (browser) side. For client side measurements I have used Firefox 8.0, and before each measurement, all cached data was emptied. I use my faithful Lenovo IdeaPad Y560 with Intel Core i5 and 8GB RAM. Testing is done on 4 most used pages in WordPress: dashboard, posts list, post editor and comments list. Other pages are similar to these 4, and they will not show much difference in overall results.

If you compare results in this benchmark with previous one comparing 3.0 vs 3.1 vs 3.2, you will see that results for these 3 versions are now different. Difference comes from using latest versions of each WordPress, using differently configured server, and new browser. It is interesting to see how much faster is Firefox 8.0 over 5.0 used for old benchmark.

Server Side Execution

For client side I have measured 4 elements: used PHP memory to generate the page, number of SQL queries executed, time server needed to generate the page and number of hooks (both actions and filters) registered for execution. As with any version of WordPress, each new version uses more memory than previous one. But, while first 3 tested versions had number of executed SQL queries in decline, new WordPress 3.3 uses 4 to 9 queries per page more. Number of queries in the test includes 3-4 SQL queries executed by GD Press Tools Pro for security purposes, and it is the same for all 4 tested versions.

With all this, execution time on server-side for WordPress 3.3 is about 10%-15% higher than in earlier version. Number of hooks is expected to rise, just like the used memory. But again, there is a big jump with new WordPress. You can see all data on the charts below.

Server Side: Used PHP Memory

Server Side: Number of SQL Queries

Server Side: Execution Time

Server Side: Attached Hooks

What this mean? In terms of server performance WordPress 3.3 is a big step backward from WordPress 3.2. I am not going into analysis why this happened, or if it was really needed, but the fact is that new WordPress doesn’t add any feature that warrants such change on the server-side.

Client Side Execution

Pages generated by new WordPress are only a bit larger than before, and the same goes for JavaScript used. Interesting thing is that on post edit page new WordPress adds less JavaScript, all thanks to new editor and quick tags code. Hopefully, we will see more JavaScript improvements like this in next versions. But, execution time is another story, and overall, this new version is slower even than WordPress 3.1 in some cases. Similar results are with cached execution, and new WordPress on average is slowest in the 3.x line.

As before, Firefox even with latest stable version 8.0 is still slower than Opera 11.6 or Chrome 16/17. It is not as drastic difference as before, but with Opera or Chrome, pages are loading some 20%-30% faster.

Client Side: Page Size

Client Side: JavaScript Size

Client Side: Execution Time

Client Side: Execution Time - Cached

What these results mean for end-user page loading? The difference is not big, we are talking about half a second slower execution. That is hardly noticeable, but it proves the conclusion from the server-side benchmark: new WordPress is slower than previous one, while in the same time you will not see why that is needed, what new features are so important to justify this performance drop.

Overall performance index

Last chart shows two interesting, overall comparison indexes. First one is for server-side performance, and other for client side. They take into account individual results for each test and each page. WordPress 3.0.6 is referent version for this test and results for it are normalized to 1.0 for all pages and tests. Lower value is better.

Server Side: Speed Index

Client Side: Speed Index

As you can see, speed index for each of the pages shows that WordPress 3.2.1 remains superior to new WordPress 3.3 in terms of performance. If we want to compare overall for client and server-side, look at the last chart bellow. Overall, WordPress 3.3.0 managed to cling to second place thanks to some of the improvements, like WP Editor API and some other JavaScript changes, but it is very close to WordPress 3.1.4 performance. WordPress 3.3 is some 7% on server-side, and (too) big 14% on the client side off from WP 3.2.1.

Overall Speed Index

Website front-end

I didn’t test new WordPress on the front-end. From using it in the past few weeks while in Beta and RC, I didn’t noticed any changes. I imagine that using some of the permalinks structures will improve speed due to permalinks changes, but other than that all 4 WordPress versions in 3.x line have very similar performance on the front end.

Conclusion

I can’t end this benchmark on a positive note as I did back in July for WordPress 3.2. I can understand that WordPress is evolving and that new features can need more resources. But, in WordPress 3.2 six months ago, features and changes were followed by considerable optimization, and that resulted in faster WordPress. This time, we got no significant new features or improvements, apart from cosmetic ones (and some of them are questionable at best) and some changes that affected only some features (uploader and editor), and yet new WordPress is gone back a full year in terms of performance, and it is comparable to WP 3.1 or even WP 3.0 in some cases.

I am sure that some developers will say that these results are not important. I agree that half a second is not much in terms of speed and that most users will not even notice the difference, but it shows that current development of WordPress is not done right. Adding new features and sacrificing speed and resource usage is not a good way to go. Big part of the blame is on development cycle, that is targeting 2 major versions each year (well, we got whole 3 major versions this year alone). Because of that, testing and optimization is not high on priorities list, developers don’t have enough time for everything. Even now, 3.3 is about one month late from projected development roadmap.

I really hope that this new WordPress performance is a ‘glitch’ in the development, and that next WordPress 3.4 will get it back on track. I also wish that WP 3.4 takes more time, and not to be rushed by May or June just for sake of having new WordPress. Lets have 9 to 12 months development cycles, and I am sure that released product will be better and more appealing to end users to get with faster upgrades.

For now, WordPress 3.2.1 is clear winner in terms of performance and resource usage between all 4 major releases in 3.x line.

Edit:

Server side Memory and Hooks charts replaced with new ones where Y axis starts from 0.