Our profanity filter and the Scunthorpe problem

Comments are back on Raptware, with firm rules: nothing goes live until a person reads it, our profanity filter flags what doesn't belong, and personal details never reach the page. We built it this way because we want the site to be safe for everyone, starting with the kids who play our games.

Here's how we did it, including the day our own filter decided that "therapist" and "class" were dirty words.

Why we had removed comments

A while ago we took comments down completely. An open box where anyone can type anything is not a good place for a child, and we'd rather have no comments than careless ones.

But comments are worth it: that's where players tell us which level gave them trouble, what they'd like in the next update, or simply that they had fun. So we didn't bring them back "as before". We brought them back with a system designed for a family audience from day one.

How comments work now

• Everything is moderated. A comment doesn't appear on the page until someone at Raptware approves it.
• We can edit before approving. If a good comment mentions, say, the name of a school or a neighborhood, we remove that detail and publish the rest. No need to throw away a whole message over one slip.
• Account required. You have to sign in to comment. That keeps bots out and gives every message an author.
• Personal details are rejected, not just flagged. If someone types a phone number, an email or an address, the box removes it as soon as they finish typing it and explains why. A tired moderator could approve a phone number by mistake, and that can't be undone, so it never even reaches the review queue.
• Only your first name is shown in public. Many kids type their full name, and some Google or Facebook accounts bring it as is. "Robert" identifies nobody; a full name next to what that person wrote does.

The Scunthorpe problem

For usernames and comments we built a profanity filter in English, Spanish and Portuguese. The first version was simple: look for each banned word anywhere in the text. And then perfectly innocent words started getting caught.

"Therapist", "class", "assassin"… each one hides a word you wouldn't say at the dinner table. In Spanish, "monóculo" (monocle) and "vehículo" (vehicle) hide another one, and so does "artículo": the word for the very article you're reading.

We're not the first to trip over this. Programmers call it the Scunthorpe problem, after an English town whose residents couldn't open an AOL account in 1996 because the town's name contains a rude word.

How we solved it

• Every word says how it's matched. Short or ambiguous words only count when they stand alone. Words with no innocent reading also count inside other words.
• A list of innocent words. Therapist, Scunthorpe, the Spanish monóculo and many more are masked before searching. We mask them instead of skipping the whole text, so gluing a swear word onto an innocent one doesn't sneak it through.
• Disguises count too. A swear word written with numbers instead of letters, or buried in the middle of a username, still gets found.

While writing this post we even found a case we'd missed: in Portuguese, "veículo" (vehicle) and "espetáculo" (show) were still being flagged. It's fixed now. That's how this works: it gets better every day.

We'd rather err on the kind side

A profanity filter always gets something wrong, one way or the other: it lets something through, or it blocks something innocent. We chose carefully which way we'd rather fail in each case:

• For names, the filter blocks. That's why it's strict about what it looks for and generous with its list of innocent words: when it's wrong, someone can't use their own name, and that's deeply unfair.
• For comments, the filter only flags. The comment reaches the moderator with a yellow warning, and a person decides. Auto-rejecting swear words only teaches people to spell them differently; with a human reading, that trick doesn't work.
• If we hide a name, we tell you. When a name can't be shown in public, "####" appears instead and its owner gets a notice to change it. Nobody is left wondering what happened.

Age ratings, out in the open

A family-safe site doesn't mean all our games are for young kids. It means parents and players always know what they're looking at. That's why every game page shows its age rating, with the badge of each rating board (ESRB, PEGI and any others the game has) and its content descriptors.

If one day we publish a game with content that isn't for minors, its rating will say so clearly, on the same page as the play button. You can see it, for example, on the Animondos' Reversi page.

What's next

We're a small studio, and we built all of this ourselves: the filter, the moderation and how ratings are shown. We did it because a game site should be a place a kid can visit without anyone having to worry.

If you find an innocent word our filter is flagging, or something that slipped past it, tell us in the comments. A person will read them, we promise.

Looking for something to play right now? Check out our free online games, no download needed.