AI 'Robot' Traffic Isn't Clogging Up Your Website Yet

Ever since AI tools like ChatGPT burst onto the scene, there’s been a bit of chatter about how these new ‘AI robots’ might start crawling all over the internet. You might have heard whispers they could chew up bandwidth or even mess with your website's performance. Good news for Aussie small business owners: it looks like that's not happening much at all, at least not yet.

A recent study by Ahrefs, a company that looks at website data, found something interesting. They checked over 137,000 websites and discovered that specific files meant to tell AI bots what they can and can’t look at — called 'llms.txt' files — were almost completely ignored. A massive 97% of them received no visits from AI bots whatsoever.

This means that for the vast majority of websites, these dedicated AI visitors aren't a concern. Only a tiny fraction, about 1% of all traffic, came from these particular AI ‘retrieval’ bots. So, if you’ve been worried about AI robots slowing down your site or causing issues, you can probably put that concern aside for now.

It’s a bit like imagining a new type of customer queue forming, only to find out almost no one is actually joining it. This isn't to say AI won't become more active online in the future, but currently, the internet isn't being swamped by these specific AI crawlers. For now, your website is likely safe from this particular type of AI traffic.