The Forums on CSS-Tricks started life as phpBB sometime in 2008. There is even an ancient video of how to apply a basic skin. Sometime in 2010 I moved them to Vanilla Forums in a response to heavy spam and the general unwieldy nature of phpBB. Now in 2013, the forums have been moved yet again, this time to bbPress.
This move was inspired by a few things.
- Heavy spam again, with nowhere to turn to fight it. One plan was to force email verification, but that setting had a bug where emails would never get verified and bug users forever. I could never find a fix.
- I didn’t love the theming. It was hard for me to figure out what stuff was in what files, then when I did figure it out, you had to copy that entire file over to your theme and customize it there. That made it safe to upgrade, but also meant that upgrading didn’t do anything because you are overriding any new changes.
- The Vanilla community is fairly small and I didn’t find it particularly helpful. I’m sure they are nice folks, but I could never seem to get the help I needed.
- Every other part of this site is powered by WordPress, so bringing it all under one roof was mega appealing. bbPress is just a plugin for WordPress now, meaning an account on CSS-Tricks works for the Forums, The Lodge, and for commenting on the blog. Not to mention spam blocking via Akismet and custom controls (we were getting terrible “Kitchen” related spam, and now I just mark as Pending any new thread that has that word in it.)
The Conversion Process
bbPress ships default with an import tool. It’s in the admin under Tools > Forums > Import Forums once the plugin is activated.
It specifically works with Vanilla as well as several other popular forum software products. All I did was give it the information it needed to connect to the database, check that I wanted to import users as well, and let it run.
It moves everything into your normal WordPress database. There were over 100,000 posts in there, and it moved them all in about 4-5 hours. Normally database operations are pretty fast, but it makes sense that it takes this long in this case. It specifically only moves a little bit at a time and waits in between requests. This is a live site after all, and best not to hammer it. I did take down the Vanilla forums while this was going on, so no new posts would be made that would be lost. It did hang a few times during the process, but luckily the converter is smart. It saves its progress along the way and when you restart it, it restarts from where it left off.
Categories moved over to new “Forums” perfectly. The forums were largely ready to rock as soon as the conversion was done.
I was ready to accept that old forums URL’s would just be broken. Kind of a bummer, but not too tragic as it wasn’t ultra-common to link to threads out in the wild and Google would catch up fairly quickly. However that didn’t need to happen!
The old URL’s were like:
And the new URL’s:
That was close enough to RegEx, and Matt Dolan had the answer for me:
RewriteRule ^forums/discussion/[0-9]+/(.+)/?$ /forums/topic/$1/ [R=301,L]
Another concern was user accounts. Any user who had a Lodge account (i.e. anyone who backed me on Kickstarter or had since signed up) who also had a forums account ran the risk of having the same username on WordPress as they were using on Vanilla. My own account had the same problem.
Fortunately bbPress import is smart about this and creates the new user under username “imported__OLD_USERNAME”. Like so:
If a user expresses any confusion on this, I can easily fix it. I simply delete the “imported_” user and choose to attribute all posts to the old user.
There is a plugin for changing usernames too, if anyone ends up wanting to do that.
Passwords were the final hiccup. Of course the passwords are encrypted, so there is no way to import them from one system to the other. I just put up a notice in the forums that let people know that if they had any trouble logging in to just reset their password and it was fine (and expected).
Why use Markdown as the language for users to write comments, replies, and new threads? There are a bunch of reasons. On any system that allows users to write and post what they write, you’ll at the very least need some auto-paragraphing. If people type a bunch of text then return-return, you’ll want to display it as two paragraphs. WordPress has autop, but Markdown also does this.
The more obvious advantage to Markdown is that it makes it easier to “just type” out a response. Italic/Emphasize something by *putting asterisks around it* or **bold** it. Links are just [link](url). Quotes are just paragraphs that start with >.
But what made it very important to me is the code escaping. People post a good amount of HTML in the comments and in the forums. They shouldn’t have to go out and escape all that HTML just to post it (e.g. turn < into <). Markdown auto-escapes code that you intentionally are specifying as code (e.g. it is within `backticks` or indented 4 spaces). That is extremely helpful.
That combined with WordPress general safety stuff (e.g. stripping
<script> tags) makes it a pretty nice system.
I use the WP-Markdown plugin by Stephen Harris which works both in comments and in bbPress. I ended up finding a few bugs with my setup, so I got the help of Justin Sainton to get it working just right. At this second, there is one more little bug regarding lists in Markdown to sort out, but after that I can send anyone the patched version of WP-Markdown if they want it. Stephen is aware of the changes and thinking they might not be right for the public version.
One of the big bugs after launch was a login issue where users would log in, change pages, and appear to be logged out (as demo’d here). It seemed like a caching issue to me. I use W3 Total Cache for that, but I had that set to not cache anything in the forums or anything for any logged in user.
So far, my best guess is that it’s browser-level caching. I found this in the .htaccess:
ExpiresByType text/html A3600
…which is the content-type for the document when pages are served. So that would make sense the the browser was trying to hang onto that page. I removed it, but it looks like something that W3 Total Cache adds by default so it gets re-added when those settings are changed. I’ll see if they have any thoughts on this.
The very next day there was a bunch of “Kitchen” spam again, which was amazing to me. That tells me the spammers are very likely human beings, which also explains how they got through the CAPTCHA effortlessly.
All I had to do though was Settings > Discussion > Comment Moderation and put those words in there and it has all but stopped.
I have a variety of functionality stuff I’d like to add via Plugin, and am looking for someone who wants to take on any of these as a freelance job.
- Alteration to WP-Markdown to allow GitHub flavored Markdown.
- A plugin that allows the original poster of a thread to (or an admin) to mark a particular answer as “the answer”. That reply would get a special class so it can be styled, and the thread itself when shown in the forum list will get a special class so it can be marked as “solved” there.
- An alteration to one of the “Mark as Read” plugins which allows you to 1) mark everything as read and 2) will auto-mark a thread as read when you open it.
- An alteration to the Feature Comments plugin that makes it work with bbPress also.
- A plugin that will send a user an email notification when they are @replied in a reply.
- A plugin that will “Clean up users” – e.g. auto-delete users that have 1) no associated posts anywhere on site 2) haven’t logged in in X time 3) have no gravatar. I have a lot of spam users this would clean up.
- A plugin that would allow any users to “Report this topic” – in case it is spam or inappropriate.
- Some kind of way to allow forum moderators to see the “Pending” queue of posts that require moderation and publish/delete/spam them.
- A “Nuke” plugin – When an admin sees a spam post, they can “Nuke” it which 1) deletes it 2) deletes everything else that user has posted. 3) deletes the user.
Quick question why use discourse?
Did you mean why not use discourse?
For sure that is what I meant.
I’m pretty sure it’s still in beta. Otherwise, I think that’d be an awesome option.
It’s cool but it wouldn’t have accomplished my goal of having the whole site under one roof.
Discourse looks decent. Anyone out there tried it?
Thanks. Good to have a peek in your kitchen (sorry, couldn’t resist) and see why you made these decisions. Hope you will enjoy moderating the forum again.
This totally got caught in moderation =)
In my opinion, i like the “new” forum. It looks so clean and fresh :D
But i have a question about the bbPress login. How do you do this without the ugly wordpress admin bar? Is it a plugin?
I believe a better approach for a sufficiently large forum is to apply user flagging of comments, and upon reaching a critical mass of flags, the post should be automatically deleted (or rather, if you’re afraid, soft-deleted, and placed into a moderator’s review).
This way, you don’t need a moderator active at all time, and you don’t need someone to approve posts (a process which can take days), but rather very quickly remove spam after the act.
This approach is being very widely used on Stack Exchange sites. (They also have a point-based reputation system that gives users privileges over time (2,000 points for editing, 3,000 for closing, 10,000 for deleting, etc).
We’d like to us bbPress on our site, but would like to change the background color to blend better with the theme. We’ve looked at the bbPress plugin’s instructions, and they’re incomprehensible.
In contrast, the instructions you have presented here are quite sensible. Can you tell us how to change the background and frame colors?
You should check in with Eliott over at Advanced Custom Fields. He just moved his support forum over to BBPress and added a “mark this post as the answer” feature. Maybe he’d be willing to share the code?
I tried bbPress years ago and got bogged down by too much spam. I tried again recently in a subdomain and haven’t got barely any spam whatsoever. I do get a lot of fake registered users, but never any posts on the forum. I’ve been using it now for about 3-4 months though, so may be too early to call.
I made a similar switch from phpBB to bbPress. I really like having everything integrated, but I have run into the same shortcomings as you.
I would also like a “mark as read” option that works well.
And I want forum users to have code buttons without enabling the full visual editor. But they need the visual editor. They also need a preview.
I hope you’ll publish a future blog post when you find solutions to these issues.
Hey Chris! Great post, and thanks for the shoutout on my Username Changer!
Any updates on the caching issue? I’m planning to install bbpress and seeing a lot of info that suggests bbpress and W3TC don’t play well together…that would be a major bummer for me, as I’m pretty much wedded to W3TC.
I experienced the caching issue recently, as well. It’s a serious pain. I’ve currently got W3TC turned off, which sucks. I’ll probably be tackling the issue over the next few days because I’ve got waaaayyyyyyy too many http requests happening.
Sorry to see you leave Vanilla. The email verification issue has been resolved in more recent versions and there are a couple anti-SPAM filters that can be installed that also help. Our paid product has even more anti-SPAM features including a point-based reputation system that limits abilities and collapsing comments that get many negative reactions from other members.
You should start to realize that this is going to be the result when you let your open source product and community fester the way it has been the last couple years. I was heavily involved in making some premium plugins and decided to abandon all the work after I got a good taste of the anemic community. Surely there could have been a way for Vanilla sell premium products and maintain a health and active free community as well.
Very nice work Chris. Now all thing in css-tricks moderate with WordPress.
it’s easy to you an us.
so you need a freelance? :P
How did you customise bbpress so people could make accounts and so when they edit there accounts, it looks different.
If I may ask, are you running a MU installation? and if so, did you install BBPress on its own site or do you have everything in one WordPress install?
Great article. I am wondering what your opinion is on how well BBPress scales. I am currently working on a few projects involving vbulletin. I personally can’t stand vbulletin as a platform. Performance wise it just doesn’t cut and it wasn’t built with front end developers mind. Do you consider BBPress a platform that can handle heavy traffic ( heavy = 20,000-30,00 visitors a day ) ?
I’d like to have your alterations to tWP-markdown, please :-)
Secondly, maybe you could write up a blogpost about how you set up bbPress – you know, all the little details’n’settings? There are some rather good questions being asked in the comments above, and personally I think how you did it is pretty sweet and I would love to have a look “under the hood”.
Login with Facebook/Twitter/Github would be cool.
Just registered on forum :) I am new Css-tricks lover :)
I understand the desire to want to bring the forum and site all under one roof but I’ve found that with enough molding you can make phpBB a pretty solid spam fighter. I have some counter measures that blocks all spam except for guest posts though they are blocked from view unless approved. I could just deactivate guest posting though and block it all for good.
Occasionally some bots get through but it is no where near as bad as it used to be with phpBB.
The other reason I use phpBB is because it has addons for all of the things you mentioned on your wishlist.
Though the “under one roof” issue is only solved by bbPress certainly.
It’s been a while since I looked into bbPress. I was worried about mingling my WordPress database (around 6000 posts) with a forum database which could possibly attract a lot of spam and other mischief. I didn’t really like having all those users in the same database. Paranoid I guess, having had a the site hacked before in the past. I eventually had to switch web hosts to get it clean again. Your WordPress database must be huge! Are you using a paid service to back it up regularly? Is bbPress installed as a network site?
Glad to hear you’ve switched to bbPress, I love it! I’ve been using bbPress on a client site for awhile now (long before an importer existed…I had to map the data and write scripts, oy!) There is a newish plugin by a long time BuddyPress/bbPress contributor named iMath that will do what you want. It’s still in beta but works really well already, you can find it here => http://wordpress.org/plugins/buddy-bbpress-support-topic/
Cleaning up old users doesn’t need to be a plugin. That’s just a single line of SQL.
I predict the bbPress community growing much in the coming months / years.
Already then will initiate using it.
Thanks for sharing this news.
I hate that. That is exactly how they are building “Buddypress” also if you’ve ever looked at that. I even mentioned this to them but they were completely oblivious and didn’t think it would be a problem.
Thanks Chris, this is a good ideas. I’m using bbPress in my site. After read your post, I’m find and set up WP-Markdown Plugin for my site. But it not working well like you’re said. Can you send me the solutions? I very like your editor, simple and very nice. ( My dream :D ). Thank you so much Chris.
I am about to implement a bbpress forum on a wordpress.org blog. It is to replace a 16 year old forum that is being closed down in a few days. I have built a test site and have the forum running fine.
I did not realize that old forum archives could be imported until I read your article, thanks for mentioning how easy it is. I do not have direct contact with the designers of the current forum, but it seems similar to the bbpress format. I have poked around in the source code and I see that they are using cgi-bin to pull up the content of the posts.
Does anyone know if cgi-bin could be used with this import function on bbpress? I know very little about cgi-bin except that it is a database used in webpage presentation.
If there is a chance I can save those 16 years of text archives, I would love to do it but I have two days to figure this out. Any suggestions?
Great. Nowadays, bbpress is getting more popular. I think, It’s time to try it my one of my website. Anyway, thanks for the info.
I am also getting lots of kitchen spam….
these days spelt like k i t c h e n
anyone found a way around this to block this once and for all?
I used to get tons of that. Just put some variations of it in the Settings > Discussion > Comment Moderation area like http://cl.ly/UR7M
Very cool write up Chris. I’ve had a blog for about 8 months and just installed bbpress. Still figuring out all the quirks and trying out the MyCred plugin to add some type of gamification-point system. Getting some bugs here and there.
I was wondering, are you planning on writing any posts about css/html/js customization for bbpress? Or is that something I should strike up in your CSS forums?
I know this is an old thread, and my question may be a bit off-topic. What if someone registered for my site would like bbpress reply/post notifications sent to a different email address than the address he is registered under? Does anyone know if this is possible? And second, is there a way to add more details about the forum topic to the notification subject line? I’m currently using the plugin “bbPress Custom Topic & Reply Notifications”. Any advice would be appreciated, thank you!
Hi – I know this is an old thread but I was wondering, did you notice any difference in the system resources after the move (positive or negative). It seems like it could possibly reduce the server load moving to one system. Is that the case?
Thanks. I’ll take my answer off the air.
When I moved my phpBB forums into bbPress forums, it created a dramatic new demand for system resources. But I’m also doing this on a WordPress multisite installation.