Next
Team

Team club

Posted: 02 Jun 2025

Taken: 02 Jun 2025

23 favorites     29 comments    721 visits

See also...

Authorizations, license

Visible by: Everyone
All rights reserved
721 visits

Server load

Server load
This is the situation we are trying to deal with, high load from AI bot crawlers (left).
When that CPU utilization is under 20% the servers earn 'credit'. That credit can build to a maximum, and is used when the load is over 20%.

The high load causes the slow loading of pages, which may exceed a time setting to cause the 'Gateway' error message. But in the current extreme case, rather than coping with a short peak load, it is persistent. So all of the credit is eventually used, with the very likely display of the Gateway error, until you are lucky enough to get a page delivered.

The effect of taking the servers off-line for maintenance yesterday can be seen with the very low dip. Both in terms of the low load allowing the credit to build, and of throwing off the AI bots. Until they woke up again.

Our amateur skills are not sufficient for solving this problem and we are seeking outside help.

Nicole Merdrignac, Bergfex, Xata, Dave C and 19 other people have particularly liked this photo

29 comments - The latest ones
 David G Johnson
David G Johnson club
Many thanks to everyone - for all that you do,. Everything on 'iper' appears to be running very smooth at the moment,.. // Dj.
12 months ago.
 J. Gafarot
J. Gafarot
Thank you very much for your words and for your work,
You are very considerate.
12 months ago.
 Valeriane ♫ ♫ ♫¨*
Valeriane ♫ ♫ ♫¨* club
Merci pour votre travail ! A bientôt ! remerciements à toute l'équipe♫
12 months ago.
 Pam J
Pam J club
GO GET EM LADS !!!!
12 months ago.
 HelenaPF en longue pause
HelenaPF en longue p… club
Mille mercis pour tous les efforts que l'équipe déploie! depuis hier tout marche bien! Espérons que cela durera. Bonne semaine à toute l'équipe.
12 months ago.
 David Slater (Spoddendale)
David Slater (Spodde… club
Thank you for all your efforts. I appreciate all you do and only wish I had technical knowledge so I could help in some way.
12 months ago.
 Tanja - Loughcrew
Tanja - Loughcrew club
Danke für die Infos und Euere Bemühungen…wenn wir verstehen, worum es geht,
verstehen wir auch warum es nicht geht! Danke!
12 months ago.
 Roger (Grisly)
Roger (Grisly) club
Thank you, both for the effort and the explanation AI already appears to have much to answer for.
12 months ago.
 DOMCHO
DOMCHO club
En faisant des recherches pour bloquer les bots IA
www.ethersys.fr/actualites/20240918-bloquer-les-bots-ia
12 months ago.
*Уαɾα club has replied to DOMCHO club
That's a good indication!
12 months ago.
 Malik Raoulda
Malik Raoulda club
Un remarquable travail qui donne déjà de bons résultats..
Bonne continuation.
12 months ago.
 Keith Burton
Keith Burton club
We're not the only site suffering from AI bot crawlers.........I read today that Wikipedia is struggling as well..........and I suspect that's only the tip of the iceberg..!!

Thank you to everyone involved for your hard work in keeping our heads above the water..!!
12 months ago.
 William Sutherland
William Sutherland club
What does it take to utilize a captcha mechanism to filter out bots?
12 months ago.
 Team
Team club
www.privacyjournal.net/block-llm-crawlers
"There is a downside: no text behind a CAPTCHA can appear on a search engine. That’s why sites tend to only use CAPTCHA to protect personal accounts. However, it’s a good solution to protect parts of your website, even if you can’t cover the whole thing without getting delisted."
12 months ago.
 StoneRoad2013
StoneRoad2013 club
Thanks for the explanation - it must be very frustrating.
Hopefully a solution can be found.
12 months ago.
 Karl Hartwig Schütz
Karl Hartwig Schütz club
Herzlichsten Dank für eure Arbeit! Ich habe leider keine Ahnung:-(
12 months ago.
 William Sutherland
William Sutherland club
Yes, then it's frustrating since it's important to be featured on search engines.
12 months ago.
 Team
Team club
There's a company that provides an API to provide bot protection, at $3690 per month.
11 months ago.
 Manuel Wesser
Manuel Wesser club
Wenn wir weniger Inhalte für unangemeldete Besucher verfügbar machen reduziert sich die Menge an Daten die durch die Bots abgesaugt werden können.

Eventuell als Notlösung um für den Moment das Problem zu reduzieren, dauerhaft wäre es schade darum die Inhalte der Öffentlichkeit zu entziehen - aber eine zusammengebrochene Seite hilft ja auch Niemandem.
11 months ago.
Xata club has replied to Manuel Wesser club
Forwarded yesterday to the tech team, thanks you very much for the insight.
11 months ago.
 Fred Fouarge
Fred Fouarge club
bedankt voor de informatie-al begrijp ik er niet veel van- als ik miljonair was ,was het opgelost dan gaf ik elke maand 3.690 us dollar hiervoor uit
in het huidige geval vindt ik het idee van Manuel Wesser een prima oplossing !
11 months ago. Edited 11 months ago.
 Steve Nordby
Steve Nordby club
Are you running any log monitoring tools like fail2ban to block unwanted requests?
11 months ago.
 Bergfex
Bergfex club
Bei allem Frust, den die ständigen Bot-Angriffe verursachen, sollten wir uns auch bewusst machen, welch großes Privileg es ist, noch ein freies Internet nutzen zu dürfen. In vielen Ländern ist das längst keine Selbstverständlichkeit mehr. Staatliche Zensur war schon immer ein Mittel, um Menschen gefügig zu machen – und diese Tendenzen finden sich zunehmend auch in Teilen der sogenannten „freien Welt“.
11 months ago.
 Jocelyne Villoing
Jocelyne Villoing club
Bien lu, je persévère et surtout je remercie toute l'équipe qui se dépense sans compter pour le loisir de tous ses membres !
Amitiés.
11 months ago.
 Loose_Grip/Pete
Loose_Grip/Pete club
Thanks to the team. Keep up the good work!
11 months ago.
 Scott Holcomb
Scott Holcomb club
Good luck team.
It took me over an hour to load one photo, encountering multiple errors and failures to load. Many features were unavailable such as "add photo to album". Loading times for all functions were very, very slow. Even adding latitude/longitude for photo location that had been functioning after the recent mapping problems was dis-functional.
If the AI bots have taken over, creating service-denial, they have succeeded. Even posting this note was a lengthy process, watching an endless spinning wheel. Ipernity is now practically unusable. As a long-term member this is quite disappointing.
11 months ago.
 Dave C
Dave C club
Good luck to the team with these issues.
Hopefully a way to cure this will be found.
I agree that it is frustrating at the moment but I highly appreciate all the hard work that the team put into keeping Ipernity the best photo sharing site on the internet.
11 months ago.
 stapel
stapel club
Vielen herzlichen Dank für eure Mühe und euren Einsatz, und zur Hölle mit den Verursachern dieser Störungen.
11 months ago.
 Adrian Jones
Adrian Jones club
Thanks for the update and info, good luck in sorting this out for us.
11 months ago.

Sign-in to write a comment.