But, before you go on, here is the reminder of previous posts:

• Test environment setup [ read more ]
• Results from administration tests [ read more ]
• Results from testing blog pages [ read more ]

Administration memory usage

Administration

First batch of tests will focus on WP administration pages. I used the same setup for these tests, and comparison between two versions of WP are on this first chart. As you can see, dashboard page now uses some 5MB less memory in both clean version and with all plugins active. Main reason for this is because feeds are now loading data through AJAX only if they are needed.

Pages are now prepared by PHP much faster, and this is noticeable on, more or less, all administration pages. Generated pages sizes are mostly the same size as with WP 2.8, with few exceptions where the generated page is much smaller, like with the dashboard, because feeds are now not a part of the page. Same speedup is visible on general settings page, and now time zone settings are loading very fast.

As you can see not all pages show memory decreasing, but that is to be expected, not all parts of WP 2.8 were problem, only some of the pages needed to be fixed. But as I said, many of the improvements are visible all over the administration. For the past few days I have been using 2.8.1 on my blogs, and I am very pleased with latest changes.

It’s important to mention, that this time I had no problems with auto upgrade as I had with updating from 2.7.1 to 2.8, so that’s also a big plus for 2.8.1

Blog Pages

The other part of the test is testing of actual blog pages. Again, many things depend on the theme you use and on the loading optimizations theme might (or not) use. As I said before, very few themes right now are improved in that direction. So, I decided for this WP 2.8 against WP 2.8.1 tests to focus on default theme and number of queries executed.

Number of queries on blog pages

As for the other tests like size of the page and used memory, there is almost no difference between two versions. But, as I said in first post on the subject, WP 2.8 is already excellent as the blog pages goes.

But in most cases, WP 2.8.1 needs less queries then WP 2.8. I am not sure exactly what is done to cause this improvement, and I really don’t care, as long as WP 2.8.1 is better. And less queries it takes, means that less resources will WP use, and server will be able to serve content to more visitors in the same time.

Last week I have published similar tests for WP 2.8 and my GD Star Rating plugin, and you can see in that post how much SQL queries will that plugin add to the blog. Tests are done to show the value of cache and prefetch methods I developed into the plugin to minimize the queries.

Conclusion

Now that we have WP 2.8.1 released, there is no more reasons for you to wait with upgrading your blog(s). WP 2.8.x uses more resources then previous versions, but that is, in my opinion, good trade off for much better interface, much easier to use panels for posts and comments, better dashboard, better user interaction and most important faster blog pages that your visitors see. If WP 2.8 was problematic in terms of using too much memory, WP 2.8.1 solved all that problems and even improved in some areas. Overall, this is the best release of WP yet, and a good sign that next versions will be, as expected, even better.

I hope that I will be able to complete (soon) testing of JavaScript performance and testing of some of the plugins to complete the series of benchmark posts trying to give you a better look at WP performance.

I made few changes to each of the WP installations so that the measuring rendering time is more accurate. Also, I changed few things in GD Press Tools also, and the Arthemia theme for the next part when the benchmark will be done for the actual blog. For some of the plugins I used older versions for WP 2.6.5, as recommended by the plugins authors. All results displayed on the charts are mean value from 5 measurements. I used Opera 9.64 browser for time and memory tests, and Firefox 3.0.11 with Firebug 1.3.3 for size of the output (personally I don’t like Firefox at all).

For test configuration and setup check the previous post: Benchmark Setup. On all 3 charts, first 3 columns for each page is for WordPress without plugins (only GD Press Tools is active), and the next 3 columns are for WordPress with all 15 plugins active, list of plugins in setup article.

Why benchmarking?

The reason I made these tests is because it’s obvious that no one in WordPress development team didn’t do anything like this. In fact they even claimed that WP 2.8 is much faster, and that’s simply not true at all. Take a look at this page, and notice that one of the highlights is speed of the admin pages. Speed can be gained using Turbo with Google Gears, and in that case browser needs less time to render the pages. But Opera, Safari, Chrome and Firefox are very fast already and will render very fast anything you throw on them. Browsers can’t speed up server response time. And Gears is not an option for Opera or Safari users (hopefully we will see better Gears browser support soon), not to mention problems I had with Gears in Firefox and that I don’t plan to start using it any time soon.

WP 2.6.5 is used only as a reference, in most cases you can’t really compare it with WP 2.7 or WP 2.8 because interface and some other things are rewritten, but it’s interesting to see how WP 2.6.5 compares to latest releases in terms of performance.

Page load times

First series of tests are to determine how much time PHP and server needs to generate a single page. Some of the things were problem to measure with WP 2.8, but I will get to that later. Even when the pages are partially loaded from cache this time is more or less the same. I noticed some 10-20% drop in WP 2.8 for reloading the same page, but that’s still too high.

WordPress load times

WP 2.8 is more than 2 times slower than previous two WordPress versions. On some pages difference is even bigger. I have measured 4 pages, 3 are WP native pages, last one is Settings panel from GD Press Tools, and it’s the fastest in the group because it doesn’t load a bunch of things most WP pages do.

But, there are some pages that need much more time. General settings page takes more than 10 or 15 seconds to load, because for some reason WP developers decided that the time zone list needs to be populated from WordPress website. This list could been included in the WordPress code. Right now it gets really frustrated to change any of these settings and need to wait all that time. That’s simple unacceptable.

Used memory

Things are pretty much the same with memory usage. As you can see 15 plugins add a lot to the memory used. WP 2.6.5 uses much less memory in both cases but that’s not a big surprise.

WordPress memory usage

WP 2.7 and WP 2.8 have totally rewritten interface and because of that it needs more memory. But the problem is with WP 2.8 that more or less should have the same memory usage as WP 2.7 because the changes were not that significant to cause this difference. Dashboard is the biggest problem and it uses some 25% more memory than WP 2.7. On some pages like the the general settings I mentioned earlier in some cases I noticed memory leaks and memory use increases to 45-55 MB. On one instance loading failed meaning that it exceeded preset 64MB. And again, that’s most likely caused by the timezone code.

WordPress output size

Page size

Finally this is something where WP 2.8 is actually better than WP 2.7. Again, WP 2.6.5 pages are smaller because of the different front end, but it’s interesting to see how much the interface ‘costs’.

WP 2.8 generates smaller pages compared to WP 2.7 mostly because of better optimization of JavaScript and CSS loading and joining several files into one. And in the same time that’s one of the reasons for slower loading and increased memory usage with WP 2.8.

Conclusion

There was a discussion on Weblogtoolscollection website about using 1 or 100 plugins, and it’s obvious that number of plugins active will increase used resources and time needed for page to be rendered. Another post that will investigate this much deeper. It will take some more time to prepare but it’s coming soon.

Based on all this is clear that in terms of speed and memory usage WP 2.6.5 administration is much better than the latest WP versions. On the other hand these are not actual pages that blog visitors see, and next post present results of blog benchmark under same condition, so I am not going to give some definitive judgement right now. I know a lot of people that still use WP 2.6.5 because they can’t increase memory every time WP is released. When WP 2.8.1 is released I will do a benchmark against current WP 2.8, to see if indeed developers managed to solve memory usage increase and lower speeds.

I use WP 2.8 on all my blogs right now, but I must say that I am not happy with performance of the administration compared to older versions. Interface IS much better, and to some point I understand and accept that the performance is a trade off for the interface and better work flow. But only to a point. I will be very disappointed to see WP 2.9 some 7-8 months down the road that instead of 40MB needs 50MB or more to work.

So, tell me what you think about data presented here, and about your own experiences with last 3 versions of WP, and especially with WP 2.8. And stay tuned for more benchmark results.

All tests will be between WP 2.6.5, WP 2.7.1 and WP 2.8. Once WP 2.8.1 is released I will compare it to WP 2.8. Tests will be done on administration pages and on blog itself. First round of tests will test server side and how different factors affect each of the tested WP versions. After that I will try to determine how the browsers render different pages for different WordPress versions.

Here is the preliminary list of posts I plan to write:

• WP Benchmark: Administration [ read more ]
• WP Benchmark: Blog [ read more ]
• WP Benchmark: 2.8.1 [ read more ]
• WP Benchmark: GD Star Rating #1 [ read more ]
• WP Benchmark: GD Star Rating #2 [ read more ]

• WP Benchmark: Plugins #1 [ to be published: soon ]
• WP Benchmark: Plugins #2 [ to be published: soon ]

• WP Benchmark: JavaScript [ to be published: soon ]

Setup for all the tests is described in this post:

• WP Benchmark: Setup [ read more ]

When I publish the post, I will update this list and add the URL. GD Star Rating tests will be published on plugin’s official website. If you have any more suggestions, leave the comment.

To begin all this, I will be using 3 WordPress version. With each version you can see the size of the installation without wp-content folder:

1. WordPress 2.6.5 (4.2MB)
2. WordPress 2.7.1 (5.3MB)
3. WordPress 2.8 (6.7MB)

All 3 versions will be set the same, all will use exact duplicate of this website, including Arthemia Premium theme, all plugins, posts, comments, settings, pages and everything else:

1. Posts: 34
2. Pages: 16
3. Comments: 215

After inserting all posts, with no plugins activated, I measured the size of databases for all 3 WP installations. WP 2.8 has significantly larger Options table mostly due to the new transient settings and new feeds format. I will post another similar chart after all the tests are done. And here is the list of installed plugin on all 3 test WordPress version.

Database Sizes

1. Akismet 2.2.4
2. GD Press Tools 1.4.0
3. GD Simple Widgets 1.2.0
4. GD Star Rating 1.4.7
5. Google XML Sitemaps 3.1.4
6. HeadSpace2 3.2.28
7. MiniMeta Widget 4.2.0
8. SABRE 0.9.0
9. SEO Booster Lite 1.6.5
10. Sociable 3.1.3
11. SyntaxHighlighter Plus 1.0b2
12. TinyMCE Advanced 3.2.4
13. Twitter Tools 1.6
14. WordPress.com Stats 1.4
15. Wordpress Download Monitor 3.1.4
16. WP-Slimbox2 0.9.7.1

GD Press Tools is always active an all test blogs and with this plugin, and benchmarking code added in it (only for these tests, it will not be added as a standard plugin feature) all measurements will be taken. All tests will be done on localhost setup on MS Windows Vista SP2 64bit with XAMPP 1.7.1.

1. Apache 2.2.11
2. PHP 5.2.9
3. MySQL 5.1.33

Test machine is very fast, AMD/nVidia based with strip mode RAID arrays and a lot of RAM:

1. CPU: AMD Phenom Black Edition 9850+ (QuadCore)
2. RAM: DDR2 8GB Kingston HyperX 800MHz
3. Motherboard: Gigabyte GA-M750SLI-DS4 with nForce 750a SLI chipset
4. HDD: 2x500GB RAID1, 2x160GB RAID1

So, now you have all the basic facts, and expect first post with benchmark results in the next 3-4 days.