This is first basic performance testing for WordPress plugins in 2012, and it will include 30 plugins. They will be tested on how much resources they use to load, and not to tell you if the plugin is good or not. Plugins features and plugin code quality is not the reviewed here.
Plan was to have more, but this first monthly measurement will have 30 instead of 50 plugins. But, this list will change over time, and new plugins will be added to the mix, and some of previously tested will be removed. But, from next month I will have publicly available table with all historical data for all tested versions.
All the data displayed here may vary depending on server configuration and other factors. Data here doesn’t say anything about quality of the plugins or their usability, it is only a test of resource usage. Also, some of the plugin can use different amount of resources depending on the plugin configuration, and all tests here are done with default plugin configurations. There is no way to test complex implication of different settings or method of work for a test like this.
Measurement Process
For this first monthly test I have tested 30 plugins. All tests are done with a single installation, and WordPress 3.3 was used. Tests are done on the local server with PHP 5.3.8. All measurements are done using GD Press Tools Pro 4.3.2 plugin. Plugin has a tracker class that can be loaded from wp-config.php file to accurately capture snapshots of different loading stages (among other things). To allow capturing plugins snapshots, I made small change to WordPress core files: added snapshot call into loop that loads plugins.
To get as accurate resource usage I spent some time testing each plugin individually to measure used memory during loading and after all plugins are loaded. Some plugins load most of its files hooked to WordPress action, and not during loading of the plugin. Again, this number is not 100% accurate, but it is very close. As these tests are improved over the next few months, the numbers can vary a bit.
Plugin Grading
Each plugin is graded on a scale from 1 (worst loading optimization) to 5 (best loading optimization). This grade is not only connected to used resources, but to actual method of loading plugin uses. Plugins that got grade 1 have no loading optimization and they always load everything (or almost everything) and that uses precious server resources always, even if the plugin doesn’t need all things loaded. There is no perfect loading method, because it would be very complicated to do. But grade 5 goes to plugins that use loading optimization, that care about what WordPress is executing, and how much resources are used actually. Change column for this first test is empty (well it is set to =), and it will change with next test depending on the progress plugin developers do with their plugins.
To give plugin a grade, I have analyzed plugin loading process and I have taken into account how much resources plugin takes due to loading method it uses. Best loading optimized plugin I know is my own GD Press Tools Pro. If this plugin is to load all its parts always it would use twice as much memory and it would need much more time to load. Even with all that, I always work on further changing how the plugin loads trying to make it better. Plugins close or better to the level of optimization in GD Press Tools Pro will be graded 5 also. If the plugin uses different amount of memory on front and back end it doesn’t mean that the grade will be higher because of that.
Before we go on to the testing, read the original article from last month regarding the plugin loading process that will give the reasons for such testing and why it is important to make sure that plugin loads parts it needs only for admin side and front end:Â Measuring impact of plugins on WordPress loading.
Tested plugins basic information
30 tested plugins are listed in the table below. Beside basic data in first 5 columns, you can see if plugin adds own JavaScript, how many AJAX handlers uses, how many cron jobs can be registered by this plugin (it doesn’t mean that all will be registered always) and how many widgets plugin can add.
| Plugin | Free | Version | Released | Website | JavaScript | Ajax | Crons | Widgets |
|---|---|---|---|---|---|---|---|---|
| Adminimize | Yes | 1.7.24 | 2011.12.19. | Home | Yes | 0 | 0 | 0 |
| All In One SEO Pack | Yes | 1.6.13.8 | 2011.12.10. | Home | Yes | 0 | 0 | 0 |
| Antispam Bee | Yes | 2.3 | 2011.12.23. | Home | Yes | 0 | 1 | 0 |
| BackWPUp | Yes | 2.1.7 | 2011.12.16. | Home | Yes | 7 | 2 | 0 |
| bbPress | Yes | 2.0.2 | 2011.11.28. | Home | Yes | 2 | 0 | 5 |
| Contact Form 7 | Yes | 3.0.2.1 | 2011.12.18. | Home | Yes | * | 0 | 0 |
| Events Manager | Yes | 5.0.1 | 2012.01.02. | Home | Yes | 2 | 0 | 3 |
| FPW Category Thumbnails | Yes | 1.4.2 | 2012.01.02. | Home | Yes | 1 | 0 | 0 |
| FPW Post Instructions | Yes | 1.2.3 | 2012.01.02. | Home | Yes | 0 | 0 | 0 |
| GD Affiliate Center Pro | No | 1.3.8 | 2011.12.31. | Home | Yes | 0 | 3 | 1 |
| GD bbPress Attachments | Yes | 1.5.3 | 2011.12.28. | Home | Yes | 0 | 0 | 0 |
| GD Custom Posts And Taxonomies Tools Lite | Yes | 1.5.0 | 2011.12.28. | Home | Yes | 0 | 0 | 2 |
| GD Custom Posts And Taxonomies Tools Pro | No | 3.5.6 | 2011.12.29. | Home | Yes | 12 | 0 | 3 |
| GD Press Tools Pro | No | 4.3.2 | 2011.12.28. | Home | Yes | 30 | 7 | 0 |
| GD Unit Converter | Yes | 1.1.1 | 2011.12.12. | Home | Yes | 1 | 0 | 0 |
| Gravity Forms | No | 1.6.2 | 2011.12.04. | Home | Yes | 17 | 0 | 1 |
| JigoShop | Yes | 0.9.9.3.1 | 2011.12.28. | Home | Yes | 7 | 1 | 8 |
| Lightbox Plus | Yes | 2.4.6 | 2012.01.01. | Home | Yes | 0 | 0 | 0 |
| MarketPress Lite | Yes | 2.4.2 | 2011.12.17. | Home | Yes | 3 | 0 | 4 |
| Members | Yes | 0.2 | 2011.06.13. | Home | Yes | 0 | 0 | 2 |
| NextGEN Gallery | Yes | 1.9.1 | 2011.12.10. | Home | Yes | 10 | 0 | 3 |
| Smart Youtube PRO | Yes | 4.0.3 | 2011.12.09. | Home | Yes | 0 | 0 | 1 |
| Subscribe to Comments Reloaded | Yes | 2.0.2 | 2011.08.31. | Home | No | 0 | 0 | 0 |
| Subscribe2 | Yes | 7.0.1 | 2011.12.14. | Home | Yes | 0 | 0 | 2 |
| SyntaxHighlighter Evolved | Yes | 3.1.3 | 2011.11.17. | Home | Yes | 0 | 0 | 0 |
| TinyMCE Advanced | Yes | 3.4.5 | 2011.12.12. | Home | Yes | 0 | 0 | 0 |
| W3 Total Cache | Yes | 0.9.2.4 | 2011.08.26. | Home | Yes | 0 | 8 | 0 |
| WooCommerce | Yes | 1.3.2.1 | 2011.12.15. | Home | Yes | 17 | 0 | 14 |
| WordPress SEO by Yoast | Yes | 1.1.2 | 2011.12.16. | Home | Yes | 4 | 1 | 0 |
| WP e-Commerce | Yes | 3.8.7.4 | 2011.12.08. | Home | Yes | 7 | 1 | 8 |
All tested plugins, but one, used the internal WordPress AJAX handling. Contact Form 7 uses own handler and that is not something I can recommend. Using WP handler is best solution considering that it is already written with security concerns in mind and it is very easy to use, making plugin fit better with WP development concepts.
Resource Usage Test Results
Here is the list of all plugins in this test, and the results.
| Administration | Frontend | Grade | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Plugin | Mem. | SQL | Time | Hooks | Mem. | SQL | Time | Hooks | Change | Grade |
| Adminimize | 0.8 | 0 | 0.015 | 9 | 0.8 | 0 | 0.014 | 9 | = | 2 |
| All In One SEO Pack | 0.5 | 1 | 0.011 | 12 | 0.5 | 1 | 0.011 | 12 | = | 3 |
| Antispam Bee | 0.3 | 1 | 0.004 | 2 | 0.2 | 0 | 0.004 | 6 | = | 3 |
| BackWPUp | 0.4 | 0 | 0.006 | 9 | 0.5 | 0 | 0.007 | 9 | = | 4 |
| bbPress | 4 | 0 | 0.056 | 216 | 3.8 | 0 | 0.046 | 205 | = | 1 |
| Contact Form 7 | 0.4 | 0 | 0.009 | 17 | 0.5 | 0 | 0.007 | 16 | = | 2 |
| Events Manager | 5 | 2 | 0.07 | 75 | 3.7 | 2 | 0.052 | 64 | = | 2 |
| FPW Category Thumbnails | 0.3 | 1 | 0.004 | 9 | 0.3 | 1 | 0.004 | 1 | = | 3 |
| FPW Post Instructions | 0.1 | 0 | 0.003 | 7 | 0.1 | 0 | 0.003 | 1 | = | 4 |
| GD Affiliate Center Pro | 1.2 | 0 | 0.023 | 16 | 0.8 | 0 | 0.015 | 9 | = | 4 |
| GD bbPress Attachments | 0.2 | 0 | 0.005 | 1 | 0.2 | 0 | 0.004 | 1 | = | 3 |
| GD Custom Posts And Taxonomies Tools Lite | 1.5 | 0 | 0.019 | 13 | 1.3 | 0 | 0.016 | 7 | = | 4 |
| GD Custom Posts And Taxonomies Tools Pro | 2.3 | 0 | 0.033 | 49 | 1.6 | 0 | 0.025 | 24 | = | 4 |
| GD Press Tools Pro | 2.9 | 0 | 0.038 | 79 | 2 | 0 | 0.028 | 39 | = | 5 |
| GD Unit Converter | 0.5 | 0 | 0.011 | 4 | 0 | 0 | 0.001 | 0 | = | 5 |
| Gravity Forms | 2.9 | 0 | 0.03 | 4 | 2.9 | 0 | 0.027 | 4 | = | 2 |
| JigoShop | 3.6 | 0 | 0.059 | 148 | 2.8 | 0 | 0.044 | 105 | = | 1 |
| Lightbox Plus | 0.7 | 0 | 0.012 | 8 | 0.7 | 0 | 0.012 | 7 | = | 2 |
| MarketPress Lite | 2.4 | 0 | 0.026 | 44 | 2.4 | 0 | 0.027 | 44 | = | 1 |
| Members | 0.2 | 0 | 0.002 | 5 | 0.1 | 0 | 0.002 | 5 | = | 3 |
| NextGEN Gallery | 1.8 | 0 | 0.027 | 32 | 1.5 | 0 | 0.021 | 21 | = | 4 |
| Smart Youtube PRO | 0.7 | 0 | 0.014 | 10 | 0.6 | 0 | 0.007 | 10 | = | 3 |
| Subscribe to Comments Reloaded | 0.3 | 2 | 0.006 | 17 | 0.3 | 2 | 0.005 | 3 | = | 3 |
| Subscribe2 | 1.3 | 0 | 0.014 | 2 | 1.3 | 0 | 0.012 | 2 | = | 1 |
| SyntaxHighlighter Evolved | 0.4 | 0 | 0.004 | 1 | 0.4 | 0 | 0.005 | 1 | = | 2 |
| TinyMCE Advanced | 0.9 | 0 | 0.011 | 13 | 0.9 | 0 | 0.012 | 13 | = | 3 |
| W3 Total Cache | 1.5 | 0 | 0.02 | 46 | 0.3 | 0 | 0.005 | 19 | = | 5 |
| WooCommerce | 3.2 | 0 | 0.056 | 100 | 3.4 | 0 | 0.051 | 134 | = | 1 |
| WordPress SEO by Yoast | 1.4 | 4 | 0.022 | 22 | 0.7 | 4 | 0.013 | 19 | = | 5 |
| WP e-Commerce | 1.3 | 0 | 0.019 | 47 | 1.3 | 0 | 0.019 | 47 | = | 2 |
So, let’s start with the good. My GD Press Tools Pro is very fine tuned to load different plugin parts when needed, and it is always evolving and improving. GD Unit Converter on the original test was badly graded, but with latest version I have changed it all, and it graded 5. W3 Total Cache has very good optimization of resource usage, and loads a lot of things when needed only. Loader maybe a too complicated for my taste, but it is effective. WordPress SEO by Yoast is a great example of well written and optimized plugin.
Plugins with potential to be better are many, and all grade 3 and 4 plugins fit into this group. NextGEN Gallery is almost there, and with a bit more work, it can go to grade 5. I plan to improve my own plugins further, and work on that is in progress for some of them, so I expect them to be improved in the next tests.
Grade 2 plugins have only some elements of optimization, but they are close to grade 1. Adminimize plugin shouldn’t even load on the front end except for some elements, and that needs to be optimized. Gravity Forms is a great plugin, that needs a long way to go but at least has some sort of loading control.
As for the grade 1 plugins, the results may be surprising considering how popular some of this plugins are. WooCommerce started from JigoShop, but almost nothing is done to change awful loader JigoShop has, so they are both graded 1. Mike Jolley from WooThemes told me that they are working on improving WooCommerce so, I am looking forward to that. bbPress also has no optimization at all, and even from latest sources for 2.1 I don’t see any improvements there, and that is not a good sign.
Why is this all important?
WordPress loads plugins on both admin side and front end. But, most plugins don’t need to behave same way in both cases. Many plugins are needed only on one side. So, a plugin can be used on admin side only, and has no use on the admin side. If the plugin is not optimize to load files only when needed, it will take same resources on front end also, even so it is not needed or used, and that is a waste of server memory and time to load. Most plugins have two parts, one to be used on admin side to set the plugin, and other that works on the front end. No need to load everything on both sides, load only what the plugin needs, and it will use least amount of memory and it will work faster.
Should you care? No. And yes. If you run few plugins, than amount of memory used is not that relevant. If you run your website on VPS or dedicated server, you have much more memory available, but if you are on the shared hosting, every MB can be critical (especially if the hosting company limits it). This test is not about plugin quality. If you prefer using All In One SEO, don’t let the lower grade be a big concern because WordPress SEO plugin got higher grade. This tests show how the plugin handle resources nothing more.
Improving results
I am sure that many developers will say that these results are not important, but considering that most of the WordPress users are on shared hosting with limited memory and resources available to them, this is most important thing to have plugins they need and still have server running fine. Same thing goes for themes also. Dev4Press website uses 20-21 plugins for the last year, but with theme (my xScape Theme Framework based) and my plugins optimizations, I managed to lower memory usage from 60M last January to 40M today.
I recommend to all plugin developers to read my tutorial: How to optimize plugin loading. Most of the things from it can be applied to all plugins, and all users will benefit from developers investing extra time to optimize own plugins.
Edit 2012.01.05. – Updated grade information.
Edit 2012.01.06. – Added few more explanations and link to original loading impact article.
It would be interesting how the grades will be calculated. Are there any weighting factors?
I have updated post with more on the grades. Grading will be changed over time, and if you have suggestions, please let me know. Grade will be always based on loading and resource usage, not the actual usability or features.
Nice done.
I just hope that all the plugin’s authors can see your test in a constructive way.
One negative surprise was bbPress, specially because it comes from the big bosses. Shouldn’t they be a good role model? Anyway, it would be nice if you could have another forum plugin in our list, just for the sake of comparison.
Thanks for the comment Daniel. bbPress acts like WordPress: always load everything. WP needs to do that for obvious reasons, but bbPress doesn’t. And good idea for another forum plugin for comparison for next texs.
Simple:Press (http://simple-press.com/) is another forum plugin that you perhaps can use as comparison to bbPress?
Thanks, I will include this on for next test.
Cool, look fwd to the February edition then 🙂
Could you please post the formular, you used to compare those plugins? So far it doesn’t look objective. And could you please explain what the factors are, that you used to compare plugins that fall in completely different categories and seem, at a first glance, not comparable? Thanks.
I think I provided enough explanation to both testing and grading in the article. I have not tested plugins usability or features, or general quality, that is not the goal here. Goal is to determine if the plugin is bad (or good) in resource usage. It has nothing to do with plugin categorization compared to other plugins in the test. Each plugin is evaluated individually.
When I take “GD Press Tools Pro” and “Market Press Lite”, then they are no too far away from each other when you compare them by their values in your table. What makes the one have “Grade” 1 and the other one 5? Is the table missing information?
As I said, values in the tables are not too relevant for the grade. Grade is based on actual code that plugin has to load own parts. GD Press Tools is a huge plugin, and if there was no loading optimization it would use twice as much memory. Plugins that score 1 or 2 are loading everything they have regardless of the actual need for things they load, and that is bad optimization, that is why they scored low.
My optimization tutorial lists all the reasons why optimization is necessary and how to achieve it. Any plugin to follow that article will without any doubt score high (4 or 5). I give here example of my another plugin GD Unit Converter. In the original measurement article from december it scored very low, and now I have applied my methods of optimization and it is a different thing now.
So basically this is a code review?
Grade is review only of the code that is used to load plugin elements. I don’t get into the quality of the code, only loading.
Cannot wait for the second round of testing. I did optimize both plugins: FPW Post Instructions and FPW Category Thumbnails following your tutorial. I hope that both of them will score 5 now.
Thanks Frank! FPW Post Instructions is really useful plugin, I already added it into development list for future projects.
Hey Millan, thanks for including bbPress in your monthly performance testings. As the lead developer of the bbPress 2.x branch of code, it’d be great to get your specific opinions on what we could do to improve, as bbPress is already doing everything you outline in your plugin optimization post.
Your findings actually aren’t that surprising. bbPress loads everything up to avoid third-party plugins hitting fatal errors from not having functions and template tags available to them, with the only caveat being admin functions and UI.
I’m more curious to see how bbPress 2.x fares using your testing methods against other forum plugins and the stand-alone bbPress 1.x branch too. bbPress 2.x relies heavily on the WordPress core actions, filters, and object cache.
It’d be the most helpful (to me and the bbPress project) to move your thoughts on optimizing bbPress specifically to this ticket to keep from fragmenting the conversation in too many places.
Thanks again for putting this together and I look forward to your thoughts and feedback.
Thanks for the comment John. Yes, I know that bbPress is a bit of a problem in this regard, considering the way it should work. I am following bbPress 2.0 development for almost a year now and I think that it is a great plugin, despite using so much resources. Since I have used old bbPress 1.0 for a long while, I can say that bbPress 2.0 is better in every way, including memory usage (I still have old forum on localhost integrated with WordPress) considering how much more we have now with pluin version.
I get that plugins or theme need to be able to have access to bbPress all the time, but I think that most users are using bbPress through normal forum pages only. Some are using shortcodes to embed things, and maybe use widgets. But, I use bbPress 2.0 on this website, and I have no need for it anywhere but in the /forums/ section. And yet, it loads everywhere always. I will try to formulate in more details what sort of changes and code fragmentation can be used to make that optimized, and also maybe find some way to use PHP auto load methods. Big problem is that this will not be some quick to implement solution, but at least we can start with partial changes and improve over time.
Regarding using the magic __autoload(), sadly it’s part of PHP 5.3 and WordPress/bbPress/BuddyPress still need to support PHP 5.2. Open to seeing what improvements an autoload patch might give though, in preparation for the move to dropping PHP 5.2.
Thanks again for your help.
Yes, that is the problem, and considering that 5.3 is used on less than 10% of websites right now, we will be stuck with 5.2 for a long time. Hopefully some other solution can be used and bbPress improved. Anyway, I am great supporter of bbPress and I plan to release one more plugin for it soon, beside the one I already made for adding attachments (GD bbPress Attachments), and I am looking forward to improvements to come in the future.
Actually, __autoload() is available in earlier versions of PHP (I’m pretty sure it was in 5.0). In 5.3.0 they added the ability to catch exceptions thrown from within it, but other than that use case you should be fine to use it.
Can you add Buddypress for next performance test? great report!
BuddyPress will be included in March tests.
I’ll go ahead and note here that BuddyPress is a *much* larger plugin than any of the others being tested here. Pitting it against something small with a totally different purpose (Akismet, Members, etc…) isn’t going to be a fair comparison.
Many of the statements you make about performance and improvements are true. Just remember that it’s incredibly difficult to add completely new functionality at little to no performance cost, let alone 8 huge new features like BuddyPress provides.
Have mercy. 🙂
It is similar situation as with bbPress, a lot of things are needed all the time, and it needs to use a lot of memory by design. And since I don’t have a lot of experience with it, I don’t want to make a quick test only, so I will push it back a bit to get more time with it.
First have to say thanks for your test, the results are important for many private and also business users that installed their WordPress on shared memory based hosts.
Knowing the past, where memory was the most important cost of a personal computer, we were always looking ahead for implementations (not only serverside-based) that are more intelligent and consume less memory. Will try to use simple:press due to the performance gains. I have bookmarked your site, hopefully the next test will include some more common plugins, though its understandable that your primary look is on the performance of the GD-based plugins.
Will we have plugin testing in February?
Unfortunately, no. It will resume soon in the different way.