From the blog · 2026-11-03
An AI agent beacon tells crawlers and assistants who you are and what you sell. Why your website sits in silence, and what we built to change it.
By Conor Corcoran (HI AI Sales Design Consultants) · 3 November 2026 · 12-minute read
Originally published on HI AI Sales Design Consultants.
![]()
A note before we start: this piece was written and published with AI help and encouragement. A human led it, checked it and put his name on it.
Setting up a website and then going out looking for clients feels like a game of blind man’s buff. You’re spun round, blindfolded, and told to grab at whatever you can reach. Choose life, choose trying to get your website online, be on Reddit, be on LinkedIn, write the blog post, engage, create content. Then create more content. Get the matching leisure wear. Meanwhile your website sits there like a shop with the lights off, waiting for someone to stumble in.
There’s an old line from the poker table, often credited to Paul Newman [1][2]: if you look around the table and can’t spot the sucker, it’s you. Look around the online marketing table for a minute. The platforms are getting paid. The ad networks are getting paid. The AI companies crawling your pages are getting paid. If you can’t see who the chump is, then it’s you.
We don’t know how it’s made. Google’s own documentation describes three stages. Googlebot crawls the web by following links and reading sitemaps. Google indexes what it finds. Then it serves and ranks results when someone searches. Google says it doesn’t accept payment to crawl a site more often or to rank it higher. [3] That’s true of the organic results. It says nothing about how many people will ever see them.
Even in 2024, before AI answers spread, SparkToro found that 58.5% of US Google searches and 59.7% of EU searches ended without a click to anywhere. Only about 360 of every 1,000 US searches sent someone to the open web. [4] AI summaries have pushed that further. Pew Research Center tracked 900 US adults and found they clicked a result 8% of the time when an AI summary appeared, against 15% when one didn’t. They clicked a link inside the summary just 1% of the time. [5] And Chartbeat data in the Reuters Institute’s 2026 trends report showed Google search referrals to more than 2,500 publishers fell by a third globally in a year, and by 38% in the US. [6]
The crawlers haven’t slowed down. They’ve just stopped sending people back. Cloudflare Radar measures how many pages each AI company crawls for every visitor it refers. In September 2026, Anthropic’s ClaudeBot crawled about 549 pages per referral, OpenAI’s GPTBot about 341 and Perplexity about 2,373. Google’s figure was close to 5. [7][8] The bots read everything, and your site gets very little back.
Cory Doctorow has a word for this: enshittification. Platforms are good to their users first, then squeeze users to serve business customers, then squeeze those customers to serve themselves. [9] In April 2024 he pointed readers to Ed Zitron’s account of how it happened at Google. According to Zitron’s reading of internal emails released in the US antitrust case, in 2019 the ads and finance teams pushed a “Code Yellow” because search queries weren’t growing fast enough. Ben Gomes, then head of search, warned that search was getting “too close to the money.” By 2020 Prabhakar Raghavan, who ran ads, was running search as well. Make search a little worse, people search more, and Google sells more ads. [10][11]
A federal judge later confirmed part of the picture. In August 2024 Judge Amit Mehta ruled that Google was a monopolist in search, and found it had raised text-ad prices step by step, with the only apparent limit being “potential advertiser outcry and bad publicity.” [12] Researchers at Leipzig University and Bauhaus-University Weimar tracked 7,392 product-review searches for a year and found top-ranked pages were, on average, “more optimized, more monetized with affiliate marketing,” with signs of lower-quality text. [13]
The internet used to be fun. You made a page because you loved something, or you wanted to sell it, you described it and people found it. Now the advice is to become a content factory. Goldman Sachs counts about 50 million creators worldwide, in an economy it expects to reach $480 billion by 2027, but only about 4% of them earn more than $100,000 a year. [14] We have influencers now. Thanks, internet. I am tempted to make a test for how useful influencers are. It’s selling, so they are useful: they are sales ambassadors doing homemade infomercials that aren’t homemade and use studio-grade equipment with agents and teams behind them. That still doesn’t make them any less annoying, or stop me being ashamed to be a human in this day and age. The fakest of the fake.
Life has now turned into a Monty Python sketch where the Yorkshiremen sit around describing how hard their childhood was and who had the worst one, trying to outdo each other in hardship. The height of silliness. Listen to all the nonsense they tell people on podcasts and Insta posts about how to be successful and get on the grind daily to get ahead (see Gary V, who is not supposed to be funny but unintentionally is). Here is an average day of a successful influencer: get up two hours before you go to bed, work out like an Olympic athlete wearing 1,000 euros of gym gear, eat nothing, drink the horrible drink I’m producing, breathe, go to the toilet, be on Ozempic but not really in public, don’t drink, take drugs instead (better for the gram) and work 28 hours a day. (I am guessing they are all on drugs, because their days sound exhausting and they have to be numb not to feel the shame.) How did we get here? Why is TikTok a thing? All that silly dancing. I have about six or seven other points I could drop, but I’ll go and do a bit of aura farming elsewhere and get back to some serious LinkedIn maxxing. Laters.
All the kids now are GEOing it, generative engine optimisation: writing so that AI answers quote you. The research is real. A Princeton-led team found that changes such as adding citations, quotations and statistics could lift a source’s visibility in AI answers by up to 40%. [15] But it’s the same treadmill with a new belt, and the shortcuts don’t always work. Ahrefs looked at 137,210 sites in May 2026. Some 28% had published an llms.txt file for AI crawlers, and 97% of those files weren’t requested even once. [16] I hope this is a thing, as we were building out the AI agent beacon, and researching it further brought me down some strange rabbit holes. [17]–[23] Agent-to-agent chatting and ranking is turning into a plus ça change situation. llms.txt is not really being used. It was supposed to be the base for making the web more agentic, but it’s all still registry-based and down to how the crawlers decide to rank and quote you.
So how do you sell a product or service online now? I’m still not 100% sure. But I’m going to give it a try.
A billboard sits by the road and hopes you look up. A beacon works more like a barker at the door. When an AI agent or crawler arrives, it answers straight away: who you are, what you do, and where to do business with you. The Agent Beacon is that answer, written down in a form machines can read.
Technically, it’s a small set of machine-readable files on your website. Ours include llms.txt (a plain-text site summary), an A2A Agent Card (how agents find each other), beacon.json, agents.txt and ai.txt.
The protocols around it. These are the standards agents are being built on. [24]
We speak llms.txt and A2A today. The rest we are watching.
You don’t write the beacon yourself. We build it from a check of your own website, and you install it.
What it won’t do. Assistants mostly read your normal pages, so clear pages and schema matter most today. llms.txt is still new and few assistants use it yet. The beacon is a low-cost bet that agents will read it later, not a ranking trick.
Every company in the HI AI family has a beacon, and each is registered on The Digital Forum. Each member gets a public profile with its own files: beacon.json, an agent card, llms.txt, a plain-language agent.md and a feed.json. Agents don’t need to browse the websites. They can start from the directory, the signed registry or the profile sitemap and follow each profile to the company’s own files.
Registered is not the same as verified. The Forum checks, together with the company’s owner, that the company controls its own website and that the facts come from the company itself. A real person does the check. The Forum then signs a record of it and publishes it next to the beacon, and anyone can verify the signature with the Forum’s public key. A member that has registered but not yet been checked is shown as pending and carries no signature. Today, Job Fair AI is pending.
We asked ChatGPT, Perplexity and Gemini the same questions about ourselves on 2 and 3 November 2026. We were signed out, in Paris, with the same wording each time. This was one run per question, with Gemini on its lightest model, so it’s an informal test, not a study.
Our conclusion is that today’s beacons are not what the big assistants read. The AIs were also slow to pick up site changes. The web is far from agentic and agent-friendly, and even then it is still burdened with mistakes and hallucination.
One beacon is a signal. Many beacons that point at each other and share a signed directory are a network, and a network is easier for an agent to find than a lone file. The more companies that join, the more useful the directory is to everyone in it. If agents are crawling sites, let’s at least help them not hallucinate and get the facts right. And if all the beacons become their own nodes, our own bots should be able to chat with other websites and get what we want, without the enshittification of search everywhere. We can but try.
So what do we actually do at the Brewery? We make AI agents to one standard: checked data goes in, the work gets done, and every result is checked to be real and useful, with its cost shown. We try to make sure the agent has a real place in your workflow. Here’s what you can do there today for free on the platform:
If you run a business with a website nobody visits, we’d like you to turn the beacon on and tell us who comes knocking.
Hopefully see you all there.
Get your beacon and join the Brewery →
Also from HI AI Works: Welcome to HI AI Works Studio, a short tour of the AI Agent Brewery and the agents we build.
The First 300 Club
20% off all agent purchases for 12 months for the first 200 members, 10% for the next 100. Early access to every agent while it is in beta.
Join the First 300 Club