Dealing with Forum Spam problems

58 posts / 0 new
Last post
Dealing with Forum Spam problems

The volume of spam hitting our forums continues to increase as many of you know. Our forum system uses a double opt-in system - you must register with a valid unique email address, plus duplicate a unique graphic string. Once submitted, an email is sent to the registered email address. That email contains a link that must be clicked on to validate the registration on the site.

Currently, we do not moderate the registration process or vet a new registration signup. However, maybe it is time to consider some other means of allowing new users to register for the forums. I hate to be restrictive, but enough of you have made comments that it's time to see what else we can do.

I'm open to suggestions. In all the years of operating the forums, I've never made a public call for moderators. Maybe part of the answer is just having more "hands on deck" so to speak. Maybe, there is a better forum software system we should setup.

If you have a suggestion, please post it here. If there are one or more steps that can help, let's figure out what they might be and make some changes.

Plus, if anyone is interested in doing some moderation, please step forward and contact me.

Thanks.

Dan

Dan Sarto's picture

Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.

Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.

Edited by: Dan Sarto on 12/10/2013 - 3:50am
We already use confirmation clicks

We already have that in place. Bots and spammers get the email and respond to it, including reading and duplicating the graphic characters they need to to submit that response.

Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.

Well, I can only think of a couple things

First, they were already registered and they were dormant. Second, they know of some backdoor hack or security weekness in the system. Are you using vBulletin? Third and least likely, there could be some voodoo going on in the system. I say it in gest but truthfully, in all my years dealing with IT, sometimes, something happens and there is no logical explanation. It never happens again so once everyone is done wracking their brains, they just give up.

Dan

Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.

New Registrations vs Postings

As is the case with almost all forums, we have a small percentage of the registered base posting at any time, compared to the number of people just reading the various posts.

We get a steady state of new registrations every day, which is why I'm hesitant to increase the admin requirements which would be needed to directly moderate new registrations each and every day.

However, that might be the way to go. Now that the new year's break is behind me, I'm going to try and find a final workable solution, based on conversations held in this thread, to implement within the next week.

Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.

well

i had a good run, but it would seem that the spammers have come up with a solution

i kind of got the impression that the spam i just got was from a person and not a script because it didnt post in all forums

and it didnt reply to all threads just a single thread in a select few forums.

and they were in no particular order and the threads were not the most recent either

really random

i really dont want to have to lock my forums down, but it just doesnt look professional to have topics on britney spears getting in the anus

im using phpbb

www.EvilAsSin.com
for more movies and downloads

the fastest polygon in the west!

YEEEEEHHHHAAAAA!

Has anyone else gotten spam in a private message? I got one yesterday, and was just wondering.

Dan, I saved the thing in case you want to address it.

Has anyone else gotten spam in a private message? I got one yesterday, and was just wondering.

Dan, I saved the thing in case you want to address it.

Yep, me too.

I got one from a "poster" here named "abcd" spaming some site. Like I'm going to visit it, eh?
I have a special jar of choice, vintage, fermented contempt I'm tempted to break open for this occasion.

I've toggled off my email-from-others function in my user control panel--'cept for the admins--just to avoid any upcoming trend in spam from posting here. That means anyone wishing to contact me ( unless they already have) will just have to do it here in the forums. C'est la vie.

"We all grow older, we do not have to grow up"--Archie Goodwin ( 1937-1998)

I've heard that blocking certain IP-ranges (especially those from North-Korea and Russia) can help a lot in blocking spam attacks. Both countries are notorious for fostering cyber-crime.

Yeah I got one too, but this person isn't very knowledeable, poorly written site, no bugs that I could see. Just someone looking to make a buck based on poor advice. Not even familiar with the workings of bulletinboards. Someone with a new computer that was promised big bucks for creating a site and links. Sending links by PM is never going to work, if folks have their email settings right.

I think confirmation and making certain fields on the profile mandatory, would slow this whole trend down a lot. These bots don't like to fill out data. They are set up to hit boards that are wide open and don't require confirmation or other information at the time of registration.

ABCD wasn't even using a bot. He's just some poor slob trying generate money with whatever advice he's been given.

Pat Hacker, Visit Scooter's World.

Yeah I got one too, but this person isn't very knowledeable, poorly written site, no bugs that I could see. Just someone looking to make a buck based on poor advice. Not even familiar with the workings of bulletinboards. Someone with a new computer that was promised big bucks for creating a site and links. Sending links by PM is never going to work, if folks have their email settings right.

I think confirmation and making certain fields on the profile mandatory, would slow this whole trend down a lot. These bots don't like to fill out data. They are set up to hit boards that are wide open and don't require confirmation or other information at the time of registration.

ABCD wasn't even using a bot. He's just some poor slob trying generate money with whatever advice he's been given.

Well, he's now sending the shit thru PM's.

He's got zero posts in the forums, is spamming members with a site (which I'm certainly NOT interested in) and couldn't care less about internet decorum.

That's not the kind of "member" I want around--regardless of his intentions.

And I've also just toggled off my PMs function as well. Really sad when it closes down accessibility for others, but I can still be reached via the forums here.
People will just have to speak louder.......

"We all grow older, we do not have to grow up"--Archie Goodwin ( 1937-1998)

2. Create a new group of users that duplicates the "Registered Users" group that contains all 9K+ users and use the default "Registered Users" group as a holding group for all new registrants. Default permissions would be changed so that new users could only post to a designated New Member forum. Spam Bots, which tend to post spam immediately, would only have access to that one forum. From there, they could easily be banned/removed. With the help of current moderators and some of the additional volunteers who came forward in this thread, we could easily determine viable new users and immediately change their permissions to the main usergroup. The last step would be to setup an automatic "promotion" that would automatically change those permissions after X number of days or X number of posts, or just leave people in that forum unless they demonstrate they're not spammers.

This sounds like the best option. My only concern is, are new, legitimate members going to feel left out? In the past, we've gotten quite a few new members contribute simply because they responded to a post. With this new system it doesn't sound like it would be posible for new members to respond to a post right away.

Maybe there is a way to time delay the postings? So a new member has to wait a day or two before posting?

Is there also a way to maybe have something like a holding tank for new members' posts that don't actually get posted untill one of the moderators approve it?

Also, I don't know how automated some of this spam is, since you do need an email account and have to click a link in the email in order for the registration to go through. Some of it feels like it's some one, and this is their job to spam forums.

Anyway, sorry for the late post. I'm still on vaction and mostly just pop in to delete spam.

Oh, and I'd be fine with everyone using their real names. I do think this would help to increase the civility amongst some of the members.

Aloha,
the Ape

...we must all face a choice, between what is right... and what is easy."

Plans for proceeding with some Spam attack defense

As best as I can figure, here are our most viable options for fighting forum spam:

1. Moderate all new users, requiring everyone to be "vetted" in some manner prior to getting posting access to the forums.

2. Create a new group of users that duplicates the "Registered Users" group that contains all 9K+ users and use the default "Registered Users" group as a holding group for all new registrants. Default permissions would be changed so that new users could only post to a designated New Member forum. Spam Bots, which tend to post spam immediately, would only have access to that one forum. From there, they could easily be banned/removed. With the help of current moderators and some of the additional volunteers who came forward in this thread, we could easily determine viable new users and immediately change their permissions to the main usergroup. The last step would be to setup an automatic "promotion" that would automatically change those permissions after X number of days or X number of posts, or just leave people in that forum unless they demonstrate they're not spammers.

The first option would require more ongoing admin efforts. The second option would require alot of work up front. I'm going to run these by my main developer to get his take.

Dan

Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.

I've resisted blocking IPs or domains

I've always refrained from blocking anything in any system, such as domains, IPs, words, etc., because there are legitimate instances for all of them. I think the answer for forum spam will be in the management of signups. I'm now leaning towards moderating registration. I figure if someone wants to join our forums, I can assume it's not just to post one thing that instance and never again. If it takes half a day or day to give them the OK, that's probably not going to cause too many problems.

Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.

I am not saying that this is a problems, but I could not edit my profile with out filling in the spam checker. I found that odd but I guess that now that it is filled in I am on the cool list.

Hey, how I'd like to zap the Kinraziral type bots. They piss me off big time.

Pat Hacker, Visit Scooter's World.

Hey Dan

Hey Dan,

You could give the folks doing the "moderating" - the ability to go into other forums and clean up when they see somethng is amiss.

I saw some of the spam a couple weeks ago and could only delete the Cafe- even though it was posted in EVERY forum....you know that shoe spam thingy!

Thanks,

Some moderators do have that control

Ultimately, we need to try to get in front of the problem, or we'll just always be chasing it.

Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.

Your techs are probably already familiar with most of this Dan, but here's a link:

http://blog.eflaunt.com/?p=18

Pat Hacker, Visit Scooter's World.

Pat Hacker, Visit Scooter's World.

To fight spam, we made that mod

It's really not a big deal. For any current forum member, if they edit their profile, they will have to put in some info into the anti-spam field. It uses the same profile system as new registrants and that required field is proving invaluable in separating real members from spammers.

Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.

i have my own forums and recently just locked new users

but i have considered the route of having a proving ground type area

i either have to just accept that this proving ground is going to be mostly full of adverts for stocks, medication and ladyboys with the occasional animator or have very few new members

i went through a phase of deleting members(or rather changing their password) but it made no difference since most accounts were used just once anyway

www.EvilAsSin.com
for more movies and downloads

the fastest polygon in the west!

YEEEEEHHHHAAAAA!

I have found the following suggestion - let me know if you think it too restrictive:

- all new registrants are automatically made members of a separate group. That group can only post to a segregated forum, which we could call New Members. That would be the only forum in which a new registered member could post.

- I can create an admin rule that says "in X days" or "after X posts" in that forum, a member is automatically given access to all the forums.

- If the reg is a spam, there would be an immediate spam post. We'd see it in the New Member forum and could ban the user. If someone posts something legitimate, we can manually move them to full access before the rule kicks in.

This sounds like it would really help us out - in theory, any newly posted spam, from a spammer or bot, would only show up in the New Member forum where it will at least be segregated until cleaned out.

Comments?

Sounds like a good plan.

Pat Hacker, Visit Scooter's World.

I think this might be a good time to consider the removal of anonymity form the forum. Many others and I have been for a long time have been frustrated with the awn forum due to spam and the consistent infighting. If we are prescreening sign ups anyway to prevent spam, why not just make it mandatory to provide a first and last name as your screen name. Many very successful forums already do this, such as www.cgchar-animation.com. Forums like this have hobbyist, students, and professionals a like and are very enjoyable and productive. They manage to maintain a level of civility and professionalism because the members realize what a small industry this is, and for the most part this creates a level of self-accountability.

I'd also recommend all current users have to go through the same process, start the forum on a fresh slate. I am sure people would be willing to trade their post count for a more productive forum. CGChar went through this transition as well several years ago. The old site was archived but when old users logged in they were informed they needed to create a new account with first and last name.

www.MattOrnstein.com
Character Animator - Lucas Arts

We do protect from spiders...but

The robots.txt file is not a guaranteed defense against spiders.

Only spiders that choose to adhere to a robots.txt file voluntarily will adhere to its parameters. Spam bots don't adhere to any rules. That's why they are so difficult to get rid.

Thanks for the suggestion.

Dan

Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.

i was thinking of trying:

create an open forum with no one being allowed to create new topics. only replies to a proving ground topic.

does anyone know if spammer bots have reply functionality or are they just create new topic commands

www.EvilAsSin.com
for more movies and downloads

the fastest polygon in the west!

YEEEEEHHHHAAAAA!

i seem to have short term success with this

i created a new section called the proving ground as suggested here

then disabled new posts

all new members have to post as a reply to the main topic within to gain access

so far, with a dozen new spam accounts created, none have posted a spam reply

www.EvilAsSin.com
for more movies and downloads

the fastest polygon in the west!

YEEEEEHHHHAAAAA!

Do you think non-anonymous IDs would eliminate infighting?

I go back and forth on anonymous user accounts. Ultimately, it's not a huge deal to me. A much bigger deal to me is the constant infighting that seems to involve just a few people. The Spam problem is annoying, but to me, much less an issue then the constant flaming.

I would imagine that while having non-anonymous accounts might curtail people's desire to comment about specific industry companies and individuals, I doubt it would curtail the sniping that goes on. There is also the issue of vetting login information - people routinely provide bogus information in any registration system they join.

It's an interesting suggestion. I'm curious what other people think.

Dan

Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.

The "infighting" in these forums is NOTHING compared to other internet venues, mostly because the people doing the fighting tend to be a bit older and more mature.
Sure there's some conflicting personalities, but one can toggle a "ignore user function" and that solves most of the feather-ruffling.

On the anonymity issue: personally, I could lean towards having to disclose a identity with no objection ( but I already do that myself).
There's a couple of reasons: one--I'll know who I'm talking to directly, and if they turn out of be a fraud , then I can shun or what-have you in due course.
Using one's name, or a handle, requires a measure of personal integrity--and lacking integrity means the person isn't really worth conversing with.
My personal mandate here is to just be honest in the face of others, and take it as it comes. My personal hallucination is that this is a largely shared ideal.

Two-- The "job offers" we get tend to have catchy ( hip??) handles attached to them, and like above, it does little to establish the veracity of the offer, or the integrity of the person making same.
Requesting a "actual" name would cut down on such bogus offers, and offers made with a false name would automatically become suspect in any truly curious person's mind.

But this isn't a sticky point with me. Frauds and scammers get revealed pretty early on because we all tend to be a sensible lot anyway, and a lot of posters here have a low tolerance for BS.

I'm not afraid of industry repercussions when voicing my opinions here. If I have a conflict with another personality, I'm not expecting to working with/for them in the future anyway.
If I am thrust into that kind of situation, I'll deal with that at that time.
I doubt any of us are here to make enemies, and clashes like what happen here are pretty much just simple misunderstandings.

Sure there ARE petty people out there, but I just don't see why their pettiness need hamper one's voice here. Think before you write remains the order of the day, then, now and always.

"We all grow older, we do not have to grow up"--Archie Goodwin ( 1937-1998)

I'm not sure how many sign-ups you get in one day, and therefore I don't know how realistic this option is: All new sign-ups are only given access to a "proving ground" forum where they only need to post a response to the question "What is you're interest in animation and the AWN forums?" Any response showing a legitimate interest can then be given access to the main AWN forums.
________
Honda CBX series picture

The new system I've setup seems to be working fine

I've added a mandatory fill-in field for all new registations. All new registrations need to be verified before complete access is granted. Until them, anyone waiting for access can post in the New Members forums. Once a day we moderate the new registrations - we let in those that comment to us, ban the rest. Other than some older dormant registrations that popup now and then to post spam, there hasn't been any new spam from any new registrations from the past week.

Dan

Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.

i dont

like i said, i just created a new section which has reply only access, and so far not a sinlge spam message has been posted. the rest of the forums are completely open and unchanged

i am guessing that the spamming app which does the dirty only has the ability to post a new topic in the first forum section. since that isnt possible on mine it does not try the next one because that would be too clever for it

feel free to create an account on my forum and check it out

it works and requires no intervention from admin

www.evilassin.com/collective

www.EvilAsSin.com
for more movies and downloads

the fastest polygon in the west!

YEEEEEHHHHAAAAA!

That's basically what the plan is

We'll have all new signups only get access to a new member forum until they're either moved out by a moderator or automatically moved out based on a time frame. The spammers always post immediately, so we can more easily weed them out.

Dan

Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.

Sounds good Dan. Hopefully this will work to curb a lot of the spam.

Aloha,
the Ape

...we must all face a choice, between what is right... and what is easy."

you might want to try out the method i suggested as it doesnt involve any moderation

www.EvilAsSin.com
for more movies and downloads

the fastest polygon in the west!

YEEEEEHHHHAAAAA!

I think the changes you made are working great Dan. Since you started that intro forum, there haven't been any posts from spammers. Now I have to get used to not logging in every hour to delete spam :D

Aloha,
the Ape

...we must all face a choice, between what is right... and what is easy."

If you have to change a user's access what's the difference?

If you have to go in and change a user's access to grant them complete access, what's the difference? This way, I go in once and can quickly grant access or not. I don't have to wait for reply's etc. Alot of people signup legitimately but never really post much. If they never reply, they've never get access. I'd rather handle it more up front.

There are many ways to do this - I think the current way now is working.

Thanks for the suggestions.

Dan

Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.

The Bots do have emails and can sense graphic characters in graphic validation schemes. They tend to do their things immediately and move on. Hence, if they're initial posts (along with any new registrant's) are segregated, they're easily identified and eliminated.

The issue of Private Message spam is different. I have to look into it, but I don't think there is much you can do other to block PMs other than to block or not block - selectivity / ignoring might not be available until after you know whom it is to keep out. I'll check it out.

As far as new registrants not being able to post immediately, that's another area where no matter what scheme you try, you have to have alot of admin eyeballs to constantly separate legitimate new reg from spammers. The system doesn't separate new registrants as good or bad. It can just segregate as far as access. Most all other activities have to be taken by some one with admin access.

Dan

Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.

Victorthroe, I think I finally got your point...

I have a function in vbulletin that will do automatic "promotions" of access based on some criteria. I can set it to automatically promote to the main access group based on making a post. That new reg usergroup can be set to only allow reply posts, so assuming the spam bots can't post replies, we'll be OK and I'll have less admin work.

Worth a try...here goes nothing...

Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.

Do you require a confirmation step in registration. Where the user receives a confirmation email and has to enter through a confirmation process before posting for the first time. I would think that would slow the spam bots down.

Pat Hacker, Visit Scooter's World.

I wouldn't mind "sacrificing" anonymity, either, mostly because there isn't a whole lot to hide anymore as far as the board's "oldbies" are concerned. We know who we are.
It won't put a stop to infighting altogether because unless we can ask for blood samples for DNA verification, those who want to stir up trouble can still simply lie about their identities. As Ken D. said, though, that's what Ignore lists are for.

I also don't have a problem with real names -- my picture's even right there in my avatar and I still act the way I do sometimes. No sweat. Names are just dressing.

I'm curious though...we seem to get so few people in here...if the process is too troublesome, trying to weed out robots, will it limit the few new people we already bring in?

something wierd happened on my forum last night

my first forum section is set to 'moderator only' access for new posts and yet i had a spammer create an account and create a new topic

i created some new accounts and tried various things to repeat their post but couldnt

how did they circumvent my restrictions?

www.EvilAsSin.com
for more movies and downloads

the fastest polygon in the west!

YEEEEEHHHHAAAAA!

I think serious traffic is generated by those of us that have been around promoting AWN on other sites. And I think the type of folks that would follow help like links are more than willing to jump through a few extra hurdles.

Pat Hacker, Visit Scooter's World.

As far as new registrants not being able to post immediately, that's another area where no matter what scheme you try, you have to have alot of admin eyeballs to constantly separate legitimate new reg from spammers. The system doesn't separate new registrants as good or bad. It can just segregate as far as access. Most all other activities have to be taken by some one with admin access.

Dan

Actually confirmation clicks have slowed spam down a lot on forums. The automated systems don't like to reply, that exposes their true location. But it's up to you and your techs Dan.

Pat Hacker, Visit Scooter's World.

I think the biggest attractor or spam on our site is the amount of spiders on our site. I just counted them. We had 138 users logged on to the site, and of those, only 24 were registered members or guests. That means there were 114 spiders crawling around on the site, looking at posts and following links to all the spam sites. Thats a lot of a hell of a lot of spiders!

If I'm not mistaken spiders are deployed by the search engines to find links and keywords so when you type in a search these sites come up. Around 80% of these spiders are "Yahoo Slurp Spiders."

I know we want seach spiders on the site, but is it really necessary for one site to have 97!! of them crawling around on a single site? The other sites combined, msn, Askjeeves, and Google, have only have 17 spiders. I think thats alittle overkill by Yahoo.

I'm not sure if ther's anything we can do about this, but it just seems alittle odd to me.

Aloha,
the Ape

...we must all face a choice, between what is right... and what is easy."

About 90% of all forum fights involve the same 4 forum members.
Focus on those people rather than overhauling the entire forum.

If I remember correctly, robots.txt

http://www.robotstxt.org/wc/exclusion.html

is for that specific purpose of getting yourself out of being spidered. There's a basic amount of control involved, meaning you can be reasonably specific on who can spider and even -what- they can spider.

I see we started out with a Nokia newyear.

Pat Hacker, Visit Scooter's World.

Suggestion found in vBulletin support forum

I have found the following suggestion - let me know if you think it too restrictive:

- all new registrants are automatically made members of a separate group. That group can only post to a segregated forum, which we could call New Members. That would be the only forum in which a new registered member could post.

- I can create an admin rule that says "in X days" or "after X posts" in that forum, a member is automatically given access to all the forums.

- If the reg is a spam, there would be an immediate spam post. We'd see it in the New Member forum and could ban the user. If someone posts something legitimate, we can manually move them to full access before the rule kicks in.

This sounds like it would really help us out - in theory, any newly posted spam, from a spammer or bot, would only show up in the New Member forum where it will at least be segregated until cleaned out.

Comments?

Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.

I have found the following suggestion - let me know if you think it too restrictive:

- If the reg is a spam, there would be an immediate spam post. We'd see it in the New Member forum and could ban the user. If someone posts something legitimate, we can manually move them to full access before the rule kicks in.

This sounds like it would really help us out - in theory, any newly posted spam, from a spammer or bot, would only show up in the New Member forum where it will at least be segregated until cleaned out.

Comments?

This makes sense. The spam is automated. I've seen spam posted on these forusm that was also posted on other forums.
In some case the stuff was the exact same, in other cases it was "customized" by culling key-words from the headers of the forums in a attempt to sound legit.
Of course, when a "new" poster claim to have read the forums for a while, then asks for emails with details about the "General Discussion" you just know something is up.

The OTHER thing is the headers themselves.
A LOT of forums that offer up a free-for-all forum ( usually called General Discussions) typically get the spam all dumped there, because the title is a common one. The Animation Cafe here gets targeted a bit less because some of the spambots are programmed to look only for those key words.
Having the Forums topic in some kind of code word might also help.
For example: The Business of Animation forums could be changed to "the Biz Buzz" or something similar to thwart the generic term "business" which the spammers target.
Slang and jargon like that is less common and more specific than general keywords and that might also help quell the influx of spam.
I've notice this trait in several different forums I frequent.

Just another idea.

"We all grow older, we do not have to grow up"--Archie Goodwin ( 1937-1998)

Pages