Grow your CSS skills. Land your dream job.

WordPress Fragment Caching Revisited

Published by Guest Author

The following is a guest post by Ryan Burnette. As you'll read below, Ryan was working on a WordPress site that utilized a plugin that used the Instagram API to pull down photos. He was using it in a bit of a non-standard way that lead to lots of requests and a very slow site. In poking around at different solutions, he came across fragment caching. But unfortunately some of the information he found was outdated, so, like a good developer, he updated it. Here's the backstory and journey.

We all know web performance is important. For developers who build custom WordPress themes, however, it's pretty far down on the priority list when actually writing code. The code which renders elements on the page is usually written in the simplest, most friendly way possible using the functions that are available. This leads to code that is easily created, read, and maintained. It also leads to elements which have a very inefficient rendering process with extraneous loops and database queries.

A few extra milliseconds really start to add up. Compound this with increases in site traffic and major performance problems can arise.

Lots of really smart people have already applied their brains to this problem. The WordPress community has produced some great caching plugins. W3 Total Cache is one of them. I love them and use them frequently, but sometimes I don't need all that power. I might want to avoid configuration or have elements which aren't cache-friendly. It's also nice to keep plugins to a minimum to avoid maintenance hassles down the road.

This led to me pursue a different approach. I wanted to use a very small amount of code to cache just a few elements on the page which are too clumsy to render on every load.

Fragment Caching

When a WordPress page loads, PHP is processed and the MySQL database is queried. Sometimes a block of code makes many queries and takes a while to run. Fragment caching takes the output of a code block and stores it so for a predetermined amount of time. When the code runs, as long as the time limit hasn't elapsed, the block is ignored and the stored output is returned and printed onto the page.

Fragment caching is nothing new. WordPress core developer Marc Jaquith wrote about fragment caching. I later found a Gist that simplified Jaquith's class into a function. I forked that and modified from there.

In WordPress versions before 2.5, WP_Cache objects could be used as Jaquith's example demonstrates for persistent caching, or caching that lasts longer than one page load. The Transients API can create persistent database objects with a convenient expiration feature. My fragment caching snippet uses this method to store fragments.

Here is few lines of code can be included in the functions.php file, allowing any output to be cached as a fragment. Here's the code.

function fragment_cache($key, $ttl, $function) {
  if ( is_user_logged_in() ) {
    call_user_func($function);
    return;
  }
  $key = apply_filters('fragment_cache_prefix','fragment_cache_').$key;
  $output = get_transient($key);
  if ( empty($output) ) {
    ob_start();
    call_user_func($function);
    $output = ob_get_clean();
    set_transient($key, $output, $ttl);
  }
  echo $output;
}

The function takes three arguments:

  • Key: a simple string which identifies the fragment. Notice that the function adds a prefix to avoid colliding with other transients. You can alter the prefix by editing the function or adding a filter that matches the 'fragmentcacheprefix' tag.

  • Time to live: a time in seconds for the cache to live. I usually make use of time constants. For example, DAYINSECONDS is 86400, the number of seconds in a day. This helps those of us who are too lazy for some simple math.

  • Function: the function which creates the output. This can be anything as the examples in this post show.

Usage Examples

Using fragment caching is as easy as wrapping some HTML and PHP in function.

Here's some code that a developer might write on a WordPress site or application.

<p>Here's some HTML.</p>

<?php
// Here's some PHP
$args = array(
  'post_type' => 'my_data',
  'posts_per_page' => -1
);
$posts = get_posts($args);
foreach ( $posts as $p ) {
  echo '<pre>';
  echo get_post_meta($p,'some_meta',true);
  echo '</pre>';
}?>

<p>The PHP in this block runs and executes queries with every page load. :(</p>

Here's the same code implemented using the fragment caching snippet. Notice we're using HTML and PHP and that gets caught by the function and cached.

Let's recap the function's three arguments:

  • A tag to represent the cache. Here's a tip. If this code varies per page, concatenate the post ID into the tag to create a separate cache for each page. This would be important if the main loop is being fragment cached.
  • The timeout. I usually use WordPress time constants, but any amount of time in seconds can be used.
  • The output code itself. Notice that it's kept inside a function. This function is passed into the fragment cache function. That's right, you can pass a function as an argument in PHP.
<?php
// After
fragment_cache('my_footer', DAY_IN_SECONDS, function() { ?>

<p>Here's some HTML.</p>

<?php
// Here's some PHP
$args = array(
  'post_type' => 'my_data',
  'posts_per_page' => -1
);
$posts = get_posts($args);
foreach ( $posts as $p ) {
  echo '<pre>';
  echo get_post_meta($p,'some_meta',true);
  echo '</pre>';
}
?>

<p>And everything this block outputs will be fragment cached. :)</p>

<?php }); ?>

Examples

Here are a few examples of places where I spare my database the effort of rendering an element more often than it really needs to.

Custom Footers

The most common place where I implement this function is in a custom footer. I'll often make a footer that contains not only WordPress menus, but menus I'm generating based on the get_posts() function and additional get_post_meta() functions for each iterated post. I've found many cases where it's taking 100-200 milliseconds to render a big footer. Fragment caching makes the load time of such elements irrelevant.

Tables Of Data

WordPress has been gaining popularity as an application development platform. There's a lot of buzz about this right now. Like it or not, people are going to build apps in WordPress. This often leads to situation where what would normally be a group of database objects are stored as posts in a custom post type. Each attribute becomes a piece of meta on that post rather than an attribute of a true database object. Querying and rendering a table of data stored in this way takes a long time. Fragment caching it can solve the problem.

Embarrassingly Long Loops

There are thousands of embarrassingly long and convoluted loops out there. I've written a few of them. No matter what inefficient piece of code you have written, you can stick it in a fragment cache and it will load fast.

A Test Case

I'm the webmaster for STUDIOCRIME, a site which aggregates street art videos. WordPress provides a fantastic, simple CMS for our curators to use when posting and organizing video content for the site. The video collection pages load over 80 posts each time they viewed. Each of these iterations also queries the database for post meta.

We're also displaying a lot of content in the sidebar using a plugin authenticates and pulls data from the Instagram API. The plugin wasn't meant to be used in quite the way we're using it. Each Widget instantiates the plugin separately. This leads to very long load times.

It sure was quick and easy to build, but milliseconds here and there added up to a page which takes between 1500 and 5000 milliseconds to render. Five seconds is a long time when waiting for a web page to load.

We chose not to use a caching plugin like W3 Total Cache because decisions about how a page should load and track user data within the PHP. Page caching would keep this PHP from running.

This presented the perfect opportunity to both use fragment caching and to test the gains that can be realized by caching fragments which are slow to load.

I ran these tests using Apache Bench. Apache Bench makes one or more requests either concurrently or back-to-back and reports the time it took the web server to serve the pages. Note that without caching a single request took about three times longer to load. Compound this with multiple requests and the time it takes to get a response gets pretty high, 3 to 5 seconds. Fragment caching the slow parts of the site got the times back down and gave us the performance we needed under heavier loads.

These test show the rendering times for a single page under a concurrent load of 10, 100 and 1000 requests.

Apache Bench Test Without Caching With Caching
10 Requests 1426 ms 518 ms
100 Requests 3498 ms 658 ms
1000 Requests 5116 ms 895 ms

Happy caching!

Comments

  1. Great post – I’ll be using this for sure in my future builds and might have to go back to a few old ones!

    8:30 in the morning and I’ve already learnt something new.

    Is there a way to “clear the cache” without having to edit the function in case of code updates/fixes you want applied immediately?

    Cheers

    • You would need to hook in something to call delete_transient() -> http://codex.wordpress.org/Function_Reference/delete_transient

      I like to set-u pa hook looking for a logged in user and a query string provided on a URL. Something like example.com/my-page/?clear-the-fragment-cache

    • Julian Mallett
      Permalink to comment#

      Ryan almost has this functionality already. If you switch it around a bit you can get it to update the transient whenever the page is loaded by a logged in user:

      function fragment_cache($key, $ttl, $function) {
          $key = apply_filters('fragment_cache_prefix','fragment_cache_').$key;
          $output = is_user_logged_in() ? '' : get_transient($key);
          if ( empty($output) ) {
              ob_start();
              call_user_func($function);
              $output = ob_get_clean();
              set_transient($key, $output, $ttl);
          }
          echo $output;
      }
      

      To actually clear the entire cache, so that you don’t have to login and reload each of the changed pages, you’d have to find and delete all transients starting with fragment_cache_ or whatever your prefix is.

    • Charles Williams
      Permalink to comment#

      @Julian Mallett: You may not want to cache a fragment for logged in users if it contains different content when you’re logged in.

      This would be easily solvable by adding another parameter to the function that specifies whether or not to allow caching for logged in users.

    • Clearing the cache is why I have a filter attached to the line where the $key variable is created. I attach a randomly generated number to the key, then have a button I put in the webmaster’s admin area that says “clear cache.” By clicking it they are regenerating the randomly generated number. The old transients are cleared upon timeout. Deleting the transient works, but that would require keeping up with what transients you are creating and deleting those or some other fanciness. I just use the random number because it’s quick and easy.

      I also have a couple lines in there that bypasses the whole function if the current user is logged in. I usually develop while logged into WordPress. So, that’s why that is there. It’s easy enough to remove those lines if they are not needed.

  2. Permalink to comment#

    Great post, I know I’ll be using this one for sure! Thanks for that tid-bit.

  3. Great post! Remember though… caching shouldn’t substitute optimization! In my opinion caching should be applied when you are absolutely certain that the code, queries and http requests are as lean as it gets. If you have those down and you are still not satisfied with the overall execution time then you should apply caching. Otherwise it just feels like hiding your dirty laundry under the rug

    • PaulGagu
      Permalink to comment#

      Stratos, I got your point but I don’t think you should applied caching after….optimization, I think in sites where you have a huge amount of traffic, you should apply BOTH at the same time, and equally in an aggressive way, in these cases these extra milliseconds saves a lots of money in bandwidth and resources and also remember Google ranks pages based on the speed to load, I worked with sites in one of the biggest magazines publisher in US and in these sites a small change in a thumbnail make a difference, so we have to optimize AND caching as much as we can.

  4. Permalink to comment#

    Excellent post, but one thing… get_transient function returns boolean false if transient is expired or does not exist.

    So it should be:

    ... if (false === $output) ...
    

    instead of:

    ... if (empty($output)) ...
    
  5. paul
    Permalink to comment#

    Great post, is this still relevant when using plugins like W3 Total Cache ? or am I comparing apples and ninjas?

    • It all depends on your specific circumstance but you would still want to use something like W3 Total Cache or WP Super Cache because that will cache the whole page and not just a part of the page.

    • Permalink to comment#

      I think it is. W3TC generates HTML version of pages, and when HTML version is served, your PHP code won’t run. So nothing to worry about.

      I think this fragment cache is best used for widgets.

  6. Permalink to comment#

    This was very helpful. I will be paying far more attention to using this in the future.

  7. PaulGagu
    Permalink to comment#

    CLEVER, CLEVER, CLEVER …… only problem………………… I just read it NOW, just finished a work of 4 months to cache pieces of pages in separate way/functions, this solution would save us tons of time, thanks or sharing this, love this CSS TRICK community, thanks for existing you make developers life easier.
    Just a question if someone can help is better to use “transient” to caching or directly to a file to be included later one, I mentioned because our highly traffic sites get crazy the database requests and as soon as we start caching to files MySql’s problems stopped, and transients still use a table in the database.

  8. Absolutely, I think caching file to avoid MySql problems will not be of much benefit than the use of “transient” caching.

  9. When I use this to cache the results of a get_posts() loop on a single.php page, it returns the title and/or the featured image for the main article on the page. Any ideas on why?

    • What’s happening is that the same block of code creates different content based on the page that is being displayed. It’s totally cool to use the fragment cache in this situation. You just need a way to distinguish between pages. If I’m working with a single, then sometimes I just attach the post ID to the fragment cache’s key. Here’s an example:

      fragment_cache('my_little_snippet'.$post->ID, DAY_IN_SECTION, function () { // do some stuff here});
      
    • Jordan Lejuwaan
      Permalink to comment#

      No the get_posts() function is not dependent on the page, it returns the same for any page it is displayed on. Any other ideas?

  10. Permalink to comment#

    I dont get why youre checking for is_user_logged_in()… I guess you dont want to serve the cached chunk to a logged in user, cuz there might be sth different for a logged in person? Even for a logged in user, there are plenty of chunks that could be cached, maybe that could be a param to the func. Also, I was thinking it would be good to check against my IP (the dev IP) and never serve cached chunks to me, cuz that could cause insanity. Furthermore, it would be good to have sth like if($_get['recache'])destroyThisChunk($thechunk) Also, it might be good to destroy caches on save_post

    I like this idea a lot, i’m going to implement it in several places.

  11. Permalink to comment#

    Great stuff. I’ve had a ton off issues with a WP install on an amazon micro EC2 instance. Caching is key.

  12. No matter what inefficient piece of code you have written, you can stick it in a fragment cache and it will load fast.

    Just be vary of things like cache stampedes because if the actual data generation process is slow, that can easily lead to problems (especially in high traffic scenarios) and you may need to rethink your approach.

  13. Permalink to comment#

    Am still new to PHP.. but i like wordpress, what you write here about caching is great, i will be paying lots of attention to it later… also i like your site comment section….. something worth emulating… thanks a lot for sharing

  14. Great article.This was very helpful. Thx!

  15. Permalink to comment#

    Zack Tollman also has a nice read on fragment caching: http://tollmanz.com/partial-page-templating-in-wordpress/

  16. Permalink to comment#

    Good stuff Ryan,

    If you really want to take the transients API to the next level I’d suggest looking into using an object caching backend like APC to bypass options table database calls entirely.

    The object-cache.php file in wp-content is the key to this.

  17. Rock
    Permalink to comment#

    This was very helpful.And the wordpress fragment caching revisited is gave a few ideas to me.
    This type of caching i used vanan transcription. If u want more ideas check it.http://www.bengali-transcription.com/About-us.php

  18. Permalink to comment#

    This is exactly what I was looking for, thanks Ryan.

    Could I ask you to elaborate on exactly how you update the key with the random number? I have created a ‘clear cache’ page which runs a function, but I can’t seem to figure out how to associate this with the key inside the fragment_cache() function.

    • I hook a function to “fragment_cache_prefix” like so.

      function my_prefix() {
          return get_option('my_random_prefix');
      }
      add_action('fragment_cache_prefix', 'my_prefix');
      

      This grabs a number which was previously stored into the WordPress options and the filter in the fragment cache function adds it to the key being used to store the transient.

      Elsewhere you can hook the following function to a button in the back end, or hook it to the “save_post” action to clear the cache when webmasters save their content.

      function clear_cache() {
          $random = rand(0,999);
          update_option('my_random_prefix',$random);
      }
      add_action('save_post','clear_cache');
      

      There are other ways to go about this, but the idea is that when the random number changes, any transients previously stored are abandoned. They’ll expire and will be cleared from the options table later on so there’s no need to manually trash them.

      This is example code so you’ll want to change the names to protect the innocent if you implement it. ;)

  19. Permalink to comment#

    This is interesting stuff. I just read this article, which talks about how transients can be cleared out before their expire time, and that the expire time is a maximum, not a minimum. Any thoughts on that?

    Also, you mention Mark Jaquith’s fragment caching article, which I was just looking at and playing around with. I’m wondering if you can talk a bit more about how fragment caching compares to using transients.

    As I’m understanding it: Mark’s method, which uses WP_Object_Cache, does not store the data in a persistent cache, where the transients will be persistent.

    I’m looking into this more these days because I like to use Advanced Custom Fields to highly customize the admin area of my WP sites with whatever input fields the project needs.

    But I get the impression that too many of these custom fields along with the logic required loop through and output their data on the front end gets expensive in terms of the number of queries to the database. So it seems that using caching like this to store blocks of code that are outputting ACF field data could help.

This comment thread is closed. If you have important information to share, you can always contact me.

*May or may not contain any actual "CSS" or "Tricks".