Schneider-Fotografie wrote:

...

Hi Nathan,

you almost did it right, but the whitelisted_bots.txt file only works if your app’s User-Agent exactly contains “Dart” or “Piwigo”. Many apps use different User-Agent strings.

....

Best regards,
Olaf

I'm embarrassed to say this but I guess I have to report so it doesn't mess anyone else up.  The lines I said I wrote in the whitelisted_bots.txt file actually worked, i.e.

Dart
Piwigo

The reason it wasn't working for me before is that I managed to misspell Dart.  I had the word Data in my file rather than Dart.

I checked the log file and it reports various strings along with:

"BLOCKED: Missing referer and cookies" and "UA: Dart/3.6 (dart:io)"

As such, I believe either of the following in whitelisted_bots.txt would have worked:

Dart
or
Dart/

--Nathan

linkingpin wrote:

Hi Olaf,

great plugin, thank for developing it, I was looking for some protection against bots.

Bots come and go, I was wondering, are you planning to keep the Known Bots list and the Whitelist in your plugin up to date, and with reccomendations ?

It looks like the whitelist is checked before the blocklist ?
So what if I block a bot, and in a next version of the plugin it shows up in your default whitelist, which setting prefers ?

Hi linkingpin,

I was really annoyed by bots too – but now I’ve got everything under control!

That’s what led me to develop thisBot  system.

To your questions:
The whitelist is where you put search engines and IPs that you explicitly want to allow – and you can remove them at any time.

The bot list is for bots you want to block – and of course, you can adjust that whenever needed.

If you want more information about bots, feel free to visit my website or just get in touch:

www.blockyoubots.com - Block You Bots
„Bots denied. Humans welcomed." :-)

Is there a way to set a limit on the size of the logfile ?

After 24 hours it's size is 234 MB, Total bot blocks: 1119

I do not have access to that particular server and can't install something like logrotate.

How far can a bot penetrate into Piwigo ?

With no speficic restrictions in the webserver, not in htaccess, not by modbits and without this plugin, can a bot traverse through the Piwigo galleries dictionary leeching our photos ?

I observed an oddity with the Bot Protection 1.1 plugin I installed, additionally to _data/bot_protection/ it created a plugins/rv_akismet/_data/bot_protection/ directory with similar initial content, and updates plugins/rv_akismet/_data/bot_protection/visitor_stats.json there. There seems to be some "current working directory" issue in some context likely when [extension by rvelices] RV Akismet is running.

linkingpin wrote:

Hi Olaf,

would it be possible to use a wildcard in the blocked filter list ?

As an example, I am now blocking some bots with almost identical names like:
SemrushBot
SemrushBot-BA
SemrushBot-BM
SemrushBot-COUB
SemrushBot-CT
SemrushBot-SI
SemrushBot-SWA
SemrushBot/7~bl

Could you adapt your plugin that one entry SemrushBot* would be sufficient to block all those bots ?

Hi linkingpin,

thank you for your suggestion! Starting with the new Bot Protection v1.0.2 (Preview) you can now use wildcards (*) in the blocklist.

Example:
SemrushBot* will automatically block all variants like SemrushBot-BA, SemrushBot-SWA, or SemrushBot/7~bl.

The preview version v1.0.2 is available for free in the Free Downloads section at www.blockyoubots.com

Your feedback is very welcome!

Olaf

Hi Olaf,

I haven't been able to find the time to install the new version of your plugin yet because I had a small issue with my Piwigo installation that I wanted to solve first.

However, I now have the idea that the problem with my Piwigo could be related to the bot protection plugin.

Would you mind looking into it? I've described it on the Piwigo Forum.
.

Yes that did the trick, thanks.

Will you include this in your latest version of your plugin ?
.