Broken Link Report on external links and Flooding

(Serge) #1

Matrix Version:

Hi,

When running a links report on external links, it’s bound to create a lot of false positives due to anti-flooding measures built in external firewalls.

For instance, if our site has 200 total links to cnet.com and the report generating tool does 200 simultaneous requests to cnet.com/about, we can trip the anti-flood measure and get blocked / no response / access denied / flooding message (depending on their firewall setup).

Is there a way to tell the report generating tool (are you guys using cURL?) to induce a delay between subsequent requests to the same remote host? That would avoid tripping the anti flood measure and get a report that is actually accurate without all the false positives.

Thanks

(Bart Banda) #2

Good idea, we currently don’t do it as most of users probably don’t hit that problem, but I can see it being beneficial.

The links report tool is very limited and basic, so there are probably other improvements we can do to it as well.

I have submitted this as a feature request into our roadmap tool for now https://squizmap.squiz.net/matrix/11079 which you can access if you are a Squiz client.