This is first basic performance testing for WordPress plugins in 2012, and it will include 30 plugins. They will be tested on how much resources they use to load, and not to tell you if the plugin is good or not. Plugins features and plugin code quality is not the reviewed here.
Plan was to have more, but this first monthly measurement will have 30 instead of 50 plugins. But, this list will change over time, and new plugins will be added to the mix, and some of previously tested will be removed. But, from next month I will have publicly available table with all historical data for all tested versions.
All the data displayed here may vary depending on server configuration and other factors. Data here doesn’t say anything about quality of the plugins or their usability, it is only a test of resource usage. Also, some of the plugin can use different amount of resources depending on the plugin configuration, and all tests here are done with default plugin configurations. There is no way to test complex implication of different settings or method of work for a test like this.
For this first monthly test I have tested 30 plugins. All tests are done with a single installation, and WordPress 3.3 was used. Tests are done on the local server with PHP 5.3.8. All measurements are done using GD Press Tools Pro 4.3.2 plugin. Plugin has a tracker class that can be loaded from wp-config.php file to accurately capture snapshots of different loading stages (among other things). To allow capturing plugins snapshots, I made small change to WordPress core files: added snapshot call into loop that loads plugins.
To get as accurate resource usage I spent some time testing each plugin individually to measure used memory during loading and after all plugins are loaded. Some plugins load most of its files hooked to WordPress action, and not during loading of the plugin. Again, this number is not 100% accurate, but it is very close. As these tests are improved over the next few months, the numbers can vary a bit.
Each plugin is graded on a scale from 1 (worst loading optimization) to 5 (best loading optimization). This grade is not only connected to used resources, but to actual method of loading plugin uses. Plugins that got grade 1 have no loading optimization and they always load everything (or almost everything) and that uses precious server resources always, even if the plugin doesn’t need all things loaded. There is no perfect loading method, because it would be very complicated to do. But grade 5 goes to plugins that use loading optimization, that care about what WordPress is executing, and how much resources are used actually. Change column for this first test is empty (well it is set to =), and it will change with next test depending on the progress plugin developers do with their plugins.
To give plugin a grade, I have analyzed plugin loading process and I have taken into account how much resources plugin takes due to loading method it uses. Best loading optimized plugin I know is my own GD Press Tools Pro. If this plugin is to load all its parts always it would use twice as much memory and it would need much more time to load. Even with all that, I always work on further changing how the plugin loads trying to make it better. Plugins close or better to the level of optimization in GD Press Tools Pro will be graded 5 also. If the plugin uses different amount of memory on front and back end it doesn’t mean that the grade will be higher because of that.
Before we go on to the testing, read the original article from last month regarding the plugin loading process that will give the reasons for such testing and why it is important to make sure that plugin loads parts it needs only for admin side and front end: Measuring impact of plugins on WordPress loading.
Tested plugins basic information
|All In One SEO Pack||Yes||188.8.131.52||2011.12.10.||Home||Yes||0||0||0|
|Contact Form 7||Yes||184.108.40.206||2011.12.18.||Home||Yes||*||0||0|
|FPW Category Thumbnails||Yes||1.4.2||2012.01.02.||Home||Yes||1||0||0|
|FPW Post Instructions||Yes||1.2.3||2012.01.02.||Home||Yes||0||0||0|
|GD Affiliate Center Pro||No||1.3.8||2011.12.31.||Home||Yes||0||3||1|
|GD bbPress Attachments||Yes||1.5.3||2011.12.28.||Home||Yes||0||0||0|
|GD Custom Posts And Taxonomies Tools Lite||Yes||1.5.0||2011.12.28.||Home||Yes||0||0||2|
|GD Custom Posts And Taxonomies Tools Pro||No||3.5.6||2011.12.29.||Home||Yes||12||0||3|
|GD Press Tools Pro||No||4.3.2||2011.12.28.||Home||Yes||30||7||0|
|GD Unit Converter||Yes||1.1.1||2011.12.12.||Home||Yes||1||0||0|
|Smart Youtube PRO||Yes||4.0.3||2011.12.09.||Home||Yes||0||0||1|
|Subscribe to Comments Reloaded||Yes||2.0.2||2011.08.31.||Home||No||0||0||0|
|W3 Total Cache||Yes||0.9.2.4||2011.08.26.||Home||Yes||0||8||0|
|WordPress SEO by Yoast||Yes||1.1.2||2011.12.16.||Home||Yes||4||1||0|
All tested plugins, but one, used the internal WordPress AJAX handling. Contact Form 7 uses own handler and that is not something I can recommend. Using WP handler is best solution considering that it is already written with security concerns in mind and it is very easy to use, making plugin fit better with WP development concepts.
Resource Usage Test Results
Here is the list of all plugins in this test, and the results.
|All In One SEO Pack||0.5||1||0.011||12||0.5||1||0.011||12||=||3|
|Contact Form 7||0.4||0||0.009||17||0.5||0||0.007||16||=||2|
|FPW Category Thumbnails||0.3||1||0.004||9||0.3||1||0.004||1||=||3|
|FPW Post Instructions||0.1||0||0.003||7||0.1||0||0.003||1||=||4|
|GD Affiliate Center Pro||1.2||0||0.023||16||0.8||0||0.015||9||=||4|
|GD bbPress Attachments||0.2||0||0.005||1||0.2||0||0.004||1||=||3|
|GD Custom Posts And Taxonomies Tools Lite||1.5||0||0.019||13||1.3||0||0.016||7||=||4|
|GD Custom Posts And Taxonomies Tools Pro||2.3||0||0.033||49||1.6||0||0.025||24||=||4|
|GD Press Tools Pro||2.9||0||0.038||79||2||0||0.028||39||=||5|
|GD Unit Converter||0.5||0||0.011||4||0||0||0.001||0||=||5|
|Smart Youtube PRO||0.7||0||0.014||10||0.6||0||0.007||10||=||3|
|Subscribe to Comments Reloaded||0.3||2||0.006||17||0.3||2||0.005||3||=||3|
|W3 Total Cache||1.5||0||0.02||46||0.3||0||0.005||19||=||5|
|WordPress SEO by Yoast||1.4||4||0.022||22||0.7||4||0.013||19||=||5|
So, let’s start with the good. My GD Press Tools Pro is very fine tuned to load different plugin parts when needed, and it is always evolving and improving. GD Unit Converter on the original test was badly graded, but with latest version I have changed it all, and it graded 5. W3 Total Cache has very good optimization of resource usage, and loads a lot of things when needed only. Loader maybe a too complicated for my taste, but it is effective. WordPress SEO by Yoast is a great example of well written and optimized plugin.
Plugins with potential to be better are many, and all grade 3 and 4 plugins fit into this group. NextGEN Gallery is almost there, and with a bit more work, it can go to grade 5. I plan to improve my own plugins further, and work on that is in progress for some of them, so I expect them to be improved in the next tests.
Grade 2 plugins have only some elements of optimization, but they are close to grade 1. Adminimize plugin shouldn’t even load on the front end except for some elements, and that needs to be optimized. Gravity Forms is a great plugin, that needs a long way to go but at least has some sort of loading control.
As for the grade 1 plugins, the results may be surprising considering how popular some of this plugins are. WooCommerce started from JigoShop, but almost nothing is done to change awful loader JigoShop has, so they are both graded 1. Mike Jolley from WooThemes told me that they are working on improving WooCommerce so, I am looking forward to that. bbPress also has no optimization at all, and even from latest sources for 2.1 I don’t see any improvements there, and that is not a good sign.
Why is this all important?
WordPress loads plugins on both admin side and front end. But, most plugins don’t need to behave same way in both cases. Many plugins are needed only on one side. So, a plugin can be used on admin side only, and has no use on the admin side. If the plugin is not optimize to load files only when needed, it will take same resources on front end also, even so it is not needed or used, and that is a waste of server memory and time to load. Most plugins have two parts, one to be used on admin side to set the plugin, and other that works on the front end. No need to load everything on both sides, load only what the plugin needs, and it will use least amount of memory and it will work faster.
Should you care? No. And yes. If you run few plugins, than amount of memory used is not that relevant. If you run your website on VPS or dedicated server, you have much more memory available, but if you are on the shared hosting, every MB can be critical (especially if the hosting company limits it). This test is not about plugin quality. If you prefer using All In One SEO, don’t let the lower grade be a big concern because WordPress SEO plugin got higher grade. This tests show how the plugin handle resources nothing more.
I am sure that many developers will say that these results are not important, but considering that most of the WordPress users are on shared hosting with limited memory and resources available to them, this is most important thing to have plugins they need and still have server running fine. Same thing goes for themes also. Dev4Press website uses 20-21 plugins for the last year, but with theme (my xScape Theme Framework based) and my plugins optimizations, I managed to lower memory usage from 60M last January to 40M today.
I recommend to all plugin developers to read my tutorial: How to optimize plugin loading. Most of the things from it can be applied to all plugins, and all users will benefit from developers investing extra time to optimize own plugins.
Edit 2012.01.05. – Updated grade information.
Edit 2012.01.06. – Added few more explanations and link to original loading impact article.