The Move to bbPress

The Forums on CSS-Tricks started life as phpBB sometime in 2008. There is even an ancient video of how to apply a basic skin. Sometime in 2010 I moved them to Vanilla Forums in a response to heavy spam and the general unwieldy nature of phpBB. Now in 2013, the forums have been moved yet again, this time to bbPress.

This move was inspired by a few things.

• Heavy spam again, with nowhere to turn to fight it. One plan was to force email verification, but that setting had a bug where emails would never get verified and bug users forever. I could never find a fix.
• I didn’t love the theming. It was hard for me to figure out what stuff was in what files, then when I did figure it out, you had to copy that entire file over to your theme and customize it there. That made it safe to upgrade, but also meant that upgrading didn’t do anything because you are overriding any new changes.
• The Vanilla community is fairly small and I didn’t find it particularly helpful. I’m sure they are nice folks, but I could never seem to get the help I needed.
• Every other part of this site is powered by WordPress, so bringing it all under one roof was mega appealing. bbPress is just a plugin for WordPress now, meaning an account on CSS-Tricks works for the Forums, The Lodge, and for commenting on the blog. Not to mention spam blocking via Akismet and custom controls (we were getting terrible “Kitchen” related spam, and now I just mark as Pending any new thread that has that word in it.)

The Conversion Process

bbPress ships default with an import tool. It’s in the admin under Tools > Forums > Import Forums once the plugin is activated.

It specifically works with Vanilla as well as several other popular forum software products. All I did was give it the information it needed to connect to the database, check that I wanted to import users as well, and let it run.

It moves everything into your normal WordPress database. There were over 100,000 posts in there, and it moved them all in about 4-5 hours. Normally database operations are pretty fast, but it makes sense that it takes this long in this case. It specifically only moves a little bit at a time and waits in between requests. This is a live site after all, and best not to hammer it. I did take down the Vanilla forums while this was going on, so no new posts would be made that would be lost. It did hang a few times during the process, but luckily the converter is smart. It saves its progress along the way and when you restart it, it restarts from where it left off.

The Transition

Categories moved over to new “Forums” perfectly. The forums were largely ready to rock as soon as the conversion was done.

I was ready to accept that old forums URL’s would just be broken. Kind of a bummer, but not too tragic as it wasn’t ultra-common to link to threads out in the wild and Google would catch up fairly quickly. However that didn’t need to happen!

The old URL’s were like:

https://css-tricks.com/forums/discussion/23936/opinions-on-css-frameworks

And the new URL’s:

https://css-tricks.com/forums/topic/opinions-on-css-frameworks/

That was close enough to RegEx, and Matt Dolan had the answer for me:

RewriteRule ^forums/discussion/[0-9]+/(.+)/?$ /forums/topic/$1/ [R=301,L]

Another concern was user accounts. Any user who had a Lodge account (i.e. anyone who backed me on Kickstarter or had since signed up) who also had a forums account ran the risk of having the same username on WordPress as they were using on Vanilla. My own account had the same problem.

Fortunately bbPress import is smart about this and creates the new user under username “imported__OLD_USERNAME”. Like so:

If a user expresses any confusion on this, I can easily fix it. I simply delete the “imported_” user and choose to attribute all posts to the old user.

There is a plugin for changing usernames too, if anyone ends up wanting to do that.

Passwords were the final hiccup. Of course the passwords are encrypted, so there is no way to import them from one system to the other. I just put up a notice in the forums that let people know that if they had any trouble logging in to just reset their password and it was fine (and expected).

Using Markdown

Why use Markdown as the language for users to write comments, replies, and new threads? There are a bunch of reasons. On any system that allows users to write and post what they write, you’ll at the very least need some auto-paragraphing. If people type a bunch of text then return-return, you’ll want to display it as two paragraphs. WordPress has autop, but Markdown also does this.

The more obvious advantage to Markdown is that it makes it easier to “just type” out a response. Italic/Emphasize something by *putting asterisks around it* or **bold** it. Links are just [link](url). Quotes are just paragraphs that start with >.

But what made it very important to me is the code escaping. People post a good amount of HTML in the comments and in the forums. They shouldn’t have to go out and escape all that HTML just to post it (e.g. turn < into &lt;). Markdown auto-escapes code that you intentionally are specifying as code (e.g. it is within `backticks` or indented 4 spaces). That is extremely helpful.

That combined with WordPress general safety stuff (e.g. stripping <script> tags) makes it a pretty nice system.

I use the WP-Markdown plugin by Stephen Harris which works both in comments and in bbPress. I ended up finding a few bugs with my setup, so I got the help of Justin Sainton to get it working just right. At this second, there is one more little bug regarding lists in Markdown to sort out, but after that I can send anyone the patched version of WP-Markdown if they want it. Stephen is aware of the changes and thinking they might not be right for the public version.

The Aftermath

One of the big bugs after launch was a login issue where users would log in, change pages, and appear to be logged out (as demo’d here). It seemed like a caching issue to me. I use W3 Total Cache for that, but I had that set to not cache anything in the forums or anything for any logged in user.

So far, my best guess is that it’s browser-level caching. I found this in the .htaccess:

ExpiresByType text/html A3600

…which is the content-type for the document when pages are served. So that would make sense the the browser was trying to hang onto that page. I removed it, but it looks like something that W3 Total Cache adds by default so it gets re-added when those settings are changed. I’ll see if they have any thoughts on this.

The very next day there was a bunch of “Kitchen” spam again, which was amazing to me. That tells me the spammers are very likely human beings, which also explains how they got through the CAPTCHA effortlessly.

All I had to do though was Settings > Discussion > Comment Moderation and put those words in there and it has all but stopped.

The Future

I have a variety of functionality stuff I’d like to add via Plugin, and am looking for someone who wants to take on any of these as a freelance job.

• Alteration to WP-Markdown to allow GitHub flavored Markdown.
• A plugin that allows the original poster of a thread to (or an admin) to mark a particular answer as “the answer”. That reply would get a special class so it can be styled, and the thread itself when shown in the forum list will get a special class so it can be marked as “solved” there.
• An alteration to one of the “Mark as Read” plugins which allows you to 1) mark everything as read and 2) will auto-mark a thread as read when you open it.
• An alteration to the Feature Comments plugin that makes it work with bbPress also.
• A plugin that will send a user an email notification when they are @replied in a reply.
• A plugin that will “Clean up users” – e.g. auto-delete users that have 1) no associated posts anywhere on site 2) haven’t logged in in X time 3) have no gravatar. I have a lot of spam users this would clean up.
• A plugin that would allow any users to “Report this topic” – in case it is spam or inappropriate.
• Some kind of way to allow forum moderators to see the “Pending” queue of posts that require moderation and publish/delete/spam them.
• A “Nuke” plugin – When an admin sees a spam post, they can “Nuke” it which 1) deletes it 2) deletes everything else that user has posted. 3) deletes the user.