AWS Blocks Popular SEO Tool AHref


AWS Blocks Popular SEO Tool AHref

Have you heard of the popular SEO tool AHref? Well, the internet is ablaze with chatter that it cannot access sites running on AWS. I have also heard reports of React but that could just mean NextJS running on Vercel which is just a wrapper for AWS.

When I heard this I got curious and went into WarGame’s mode. I instantly noticed that when I hit one of my domains with AWS WAF turned on the chances of the request getting blocked were much higher but not guaranteed.

So I started tracking requests and crawling the WAF logs to see if they were getting blocked. They were NOT getting blocked. Everything was Allowed.

So then I traced it in the API Gateway Access Logs. Those were going through and coming back with a response; there was no significant difference in latency or anything.

Having WAF turned on or off seemed to have a statistically significant, though not 100%, effect on whether or not the request got through.

The really funny thing is that for my new O’Reilly On Demand Course on AWS Security (Coming soon 🙂) my WAF rules were a joke. One regex rule blocks you if you go to /blockme or .htaccess so why on earth does that have a significant impact?

What would be really nice is if I could get my hands on AHref’s side of the logs and source code and do some digging there. If anyone knows anyone at AHref send them my way. I would do it out of professional curiosity.

This one is a head-scratcher. Writing about it won't quite do it justice so be sure to check out the video. Let me know if you want me to do a deeper dive.