Abuse of registration

[Adri1]

We ask People who are interested in our site to fill in a registration form.

Recently we get a lot of requests for registration from 'suspected' persons.

How can we ban them?

 

[Katryne]

Hello ! You may slow down this keen interest with installing the Image captcha mod.
If the flow is not slowed down to an honest level, you can record the IP addresses of these unwanted visitors and ban them in your htaccess file

[Rob Severijns]

15 hours ago, Adri1 said:

'suspected' persons

Do you mean bots or actual people?
I also agree with Kathryne regarding the Captcha. Could be the Google Captcha or the Image Captcha mod

[Katryne]

I'm being invaded by thousands of bot connections. This week, they were crawling via calendar.php, so I temporarily renamed this file, but now it's descend.php and pedigree.php. The bots have spotted url patterns and are building connecting links by replacing the variable with the person or family ID. It's a never-ending battle... Is it possible to limit the entry url to the home page, leaving the visitor to explore further?

[Rob Severijns]

@Katryne

I use the Restrict Access mod - TNG_Wiki to make sure only logged in users can view certain content and the 404 Error mod and bot-trap mod to block bots.

I also use the Rip Prevention Mod - TNG_Wiki to see which IP addresses tried to view/index the contents of my site. With that mod I'm able to block unwanted visitors

Maybe not entirely helpfull to you since your site is public.
Mine used to be partialy public but I went back to logged in users because I was flooded by bots too.

It seems to me that with AI being used by bots it became much harder to stop them.

Hope others have better solutions to protect public sites

[Rob Severijns]

BTW if you combine the 404 Error mod with the bot-trap mod the following happens

If Bot-trap is installed, there is an additional option to ban IPs. This feature bans bots and clients that modify the URL causing a 404 error.
Logged in users are not banned while those not logged in can unban themselves.

Hope this helps

[Katryne]

Rob, I also installed Rip Prevention and Bot-trap mods, but they are useless, though I tried several different settings.
Bot-trap blocks an IP once a month, maybe. But I am not sure it blocked any IP since Apache 2.4. (my htaccess has been using the require not instead of deny from for some times now)
As for RIP, I may have not understood the way to use it, since since the IP I ban go on been granted access.

[Rob Severijns]

I tested https://clan.chauvigne.info/ and was able to alter the url and navigate to several pages.

The 404 Error mod combined with the bot-trap mod should mitigate that if bots alter the url.

This is part of the Access log by Rip Prevention

The 166.108.197.103 being Huawei Cloud

I also question if it's Apache 2.4 being unable to stop bots effectively or that bots have become to smart and bypass .htaccess directives

[Katryne]

I will try the 404 mod. Maybe my new htaccess for Apache 2.4 is not very efficient. Before I installed it, Bot-Trap would block 20 to 30 IP per month, and none since the new htaccess : I am more than awkward with code.

[Rob Severijns]

I ran my .htaccess flie through ChatGPT and asked it to make it more efficient which it did.

Eventhough I have several bot variants of Google, Amazon and Huawei etc blocked in .htaccess they still bypass the 2.4 directives in .htaccess

[Rob Severijns]

This is what I currently have in my .htaccess file

<IfModule mod_rewrite.c>
RewriteEngine On

# Redirect HTTP to HTTPS for all requests if not secure
RewriteCond %{HTTPS} !=on
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L,QSA]

# Block specific TLDs and domains via Referer
RewriteCond %{HTTP_REFERER} \. (br|cn|ir|kp|ru|sg|xyz|amazonaws\.com|googleusercontent\.com|hwclouds-dns\.com|petalbot\.petalsearch\.com|bc\.googleusercontent\.com) [NC]
RewriteRule .* - [F,L]

# Block specific hostnames (e.g., AWS, Google Cloud, Huawei Cloud)
RewriteCond %{REMOTE_HOST} \.(ecs-.*\.compute\.hwclouds-dns\.com|ec2-.*\.amazonaws\.com|.*\.googleusercontent\.com)$ [NC]
RewriteRule .* - [F,L]

# Block common bots and scrapers by User-Agent
RewriteCond %{HTTP_USER_AGENT} "(bot|crawler|spider|facebookexternalhit|slurp|python-requests|curl|wget)" [NC]
RewriteRule .* - [F,L]

# Block requests with empty or suspicious User-Agent headers
RewriteCond %{HTTP_USER_AGENT} ^$ [OR]
RewriteCond %{HTTP_USER_AGENT} "Mozilla/4.0" [NC]
RewriteRule .* - [F,L]
</IfModule>

# Block access based on Host or Referer containing cloud-related domains
SetEnvIf Host (hwclouds|amazon) block_access
SetEnvIf Host \.compute\.hwclouds-dns\.com block_access
SetEnvIf Referer (hwclouds|amazon) block_access
SetEnvIf Referer \.compute\.hwclouds-dns\.com block_access

# Block access to specific file types from search engines
<FilesMatch "\.(docx|pdf|png|jpeg|jpg|gif|mp4|bmp|wav|mp3|wmv|mpeg|php)$">
Header set X-Robots-Tag "noindex, nofollow, nositelinkssearchbox, noimageindex, noarchive, nocache, notranslate, nosnippet, noyaca"
</FilesMatch>

# Custom logging for tracking access attempts
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{User-Agent}i\"" traplog
CustomLog /var/www/vhosts/severijns.family/httpdocs/Severijns/private/trap.txt traplog

# Rate limiting to prevent abuse
<IfModule mod_ratelimit.c>
    SetEnvIf Remote_Addr ".*" RATE_LIMIT
    <Location />
        SetOutputFilter RATE_LIMIT
        SetEnv rate-limit 5  # Allow 10 requests per second
    </Location>
</IfModule>

# Preventing DoS attacks with request timeout
<IfModule mod_reqtimeout.c>
    RequestReadTimeout header=5-10,MinRate=500 body=10,MinRate=500
</IfModule>

# Harden HTTP headers for security
<IfModule mod_headers.c>
    Header set X-Content-Type-Options "nosniff"
    Header set X-Frame-Options "SAMEORIGIN"
    Header set Referrer-Policy "no-referrer"
    Header set Strict-Transport-Security "max-age=31536000; includeSubDomains"
</IfModule>

# Deny access to sensitive files
<FilesMatch "^\\.(ht|env|config)">
    Require all denied
</FilesMatch>

    <RequireAll>
        Require all granted
        Require not env COUNTRY_CODE CN BY IR RU SG KP
        Require not env badBot
        Require not host ecs-.*\.compute\.hwclouds-dns\.com
        Require not host hwclouds-dns.com
    </RequireAll>
</IfModule>

 

[Katryne]

My htaccess too was updated to 2.4 by ChatGPT. Maybe that's the reason why it's not efficient against bots ???

# REDIRECTION HTTP vers HTTPS clan - clan been my sub-domain
RewriteCond %{HTTPS} !=on
RewriteRule .* https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
# This is your .htaccess file converted to Apache 2.4 commands
RewriteCond %{HTTPS} !=on
RewriteRule .* https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
SetEnvIfNoCase User-Agent "PetalBot" badBot
SetEnvIfNoCase User-Agent "Googlebot" badBot
SetEnvIfNoCase User-Agent "googleusercontent" badBot
SetEnvIfNoCase User-Agent "Yahoo" badBot
SetEnvIfNoCase User-Agent "msnbot" badBot
SetEnvIfNoCase User-Agent "bingbot" badBot
SetEnvIfNoCase User-Agent "MJ12bot" badBot
SetEnvIfNoCase User-Agent "Yandex" badBot
SetEnvIfNoCase User-Agent "BaiDuSpider" badBot
SetEnvIfNoCase User-Agent "AhrefsBot" badBot
SetEnvIfNoCase User-Agent "Mail.ru" badBot
SetEnvIfNoCase User-Agent "BLEXBot" badBot
SetEnvIfNoCase User-Agent "SemrushBot" badBot
SetEnvIfNoCase User-Agent "googleusercontent" badBot
SetEnvIfNoCase User-Agent "SEOkicks-Robot" badBot
SetEnvIfNoCase User-Agent "amazonaws.com" badBot
SetEnvIfNoCase User-Agent ^us-west-2.compute.amazonaws.com badBot
SetEnvIfNoCase User-Agent ^compute.amazonaws.com badBot
SetEnvIfNoCase User-Agent ^amazonaws.com badBot
SetEnvIfNoCase User-Agent ^compute-1.amazonaws.com badBot
SetEnvIfNoCase User-Agent ^us-west-2.compute.amazonaws.com badBot
SetEnvIfNoCase User-Agent ^.us-west-2.compute.amazonaws.com badBot
<RequireAll>
    Require all granted
    Require not env badBot
    # Line below was modified because you CANNOT use asterisk wildcard in IP address with Apache 2.4. However, all IPs beginning with 47.128. will be blocked
    Require not ip 47.128.
    # Line below was modified - you CANNOT use a hyphen in an IP address in Apache 2.4
    Require not ip 100.92.220.10
    Require not ip 216.244.66.249
    Require not ip 200.243
    Require not ip 114.119.132
    Require not ip 195.82.146.250
    # Anything below this line came from the Bot-Trap mod.
    Require not ip 65.109.146.28
    Require not ip 65.109.146.33
    Require not ip 104.143.89.145
    Require not ip 185.65.134.180
    Require not ip 185.213.155.139
    Require not ip 185.204.1.181
    Require not ip 146.70.225.212
    Require not ip 82.102.19.90
    Require not ip 185.213.155.174
    Require not ip 2a03:4000:47:285:a493:40ff:fe51:809f
    Require not ip 47.82.11.219
    Require not ip 185.122.141.215
    Require not ip 212.30.36.51
    Require not ip 212.30.36.34
</RequireAll>

[Adri1]

Katryne and Rob,

Is this all effective for our email adress?

They send requests for membership.

[Katryne]

If you enable a captcha for the contact form, you will get less membership requirements in your mail box. Have a look at the contact form in the menu bar on my site, you will see what it looks like.

[Adri1]

Do I understand it Well.

They are not attacking our email adres as such, but they attack us via the registration form?

[Katryne]

Is your email address displayed in plain text on your site? (Hint: it shouldn't) If not, "they" have no mean to knew it and use the contact form. Try to protect the contact form with Image Captcha mod and see if you go on receiving unwanted e-mails.

[Rob Severijns]

The e-mail address doesn't have to be displayed.
Bots will use the Register button to see if they can require access.

That's why auto approve requests should not be set to Yes

My guess is the request form Adri1 is recieveing doesn't have much (reliable) information.

Best way to mitigate this is still the Captcha option.

Adri1, can you provide us with an example of such a request?

[Rob Severijns]

Another option, based on the requests you recieve, is to set a rule in the e-mail account you use for recieving account requests.

[Adri1]

Hier is een voorbeeld.

 

 

[Katryne]

Your mail address is displayed in full view on your facebook account and the link to your facebook is too on your TNG site. And you still have not protected your contact form with captcha. Now this mail address is dead, for all the spammers in the world know it. You will have to delete it and  replace it. It's like putting a spotlight in the middle of the night to attract evil-doers.

[Rob Severijns]

Adri1,

The e-mail you recieve also has to contain an e-mail address from the requester.

Based on the Username & Password & e-mail address you can set rules in the e-mail account to block certain requests from ever reaching you.

As to Kartyne's remarks.

She's absolutely right. No Captcha installed means you will forever be flooded with these kind of requests.

Is there a specific reason why you don't us a Captcha? If I recall correct I gave you that advise months ago.
It  is the bare minimum you can do to protect yourself from those requests. The Captcha mod is the easiest one to implement.

 

[Katryne]

The Bot-trap mod must have taken offence when I called it useless, or maybe there's an upsurge in data-stealing indexing robots, because recently I've been receiving several messages a day from Bot-trap, which tells me it's doing its job very well.

[Adri1]

In the Mean time I installed the Image captcha mod 3 days ago.

Until now it seems it is doing the job and we did not receive annoying requests.

[Katryne]

Sur 15/03/2025 at 10:12 , Rob Severijns said:

The 404 Error mod combined with the bot-trap mod should mitigate that if bots alter the url.

Very efficient : the number of visitors passed from 102000 a day to 62. Not 62000, I really mean 62. I cannot find anymore trace of the least ripper robot.