Check access for useful agents and crawlers
in your robots.txt and any anti-bot protections.
AI agents do more than read the web: they are beginning to navigate websites, use forms, search for products and carry out actions on behalf of users. A website designed only for human navigation can therefore be difficult for these new visitors to understand or use. OpenAI recommends following accessibility best practices to help its agents interpret buttons, menus and forms correctly.
The first step is to make your important information easily accessible to machines. Your products, services, prices, terms and availability should be clearly structured and should not depend solely on visual elements or complex interactions. Structured data, a clear architecture, explicit content and, where appropriate, APIs or catalogues can make this information easier to understand.
You then need to consider actions. Can an agent search for a product, complete a form, request a quote or make a booking? Properly identified elements, simple journeys and accessible interfaces make these interactions more reliable. The goal is not necessarily to create dedicated infrastructure for agents, but to reduce the ambiguities that can prevent them from acting correctly.
Finally, check who can actually access your website. Search crawlers, agents acting for a user and bots used for training do not always serve the same purpose and can be treated differently. Cloudflare, for example, distinguishes between these three uses — Search, Agent and Training — so website owners can apply different rules.
in your robots.txt and any anti-bot protections.
rather than relying solely on the visual interface.
especially buttons, forms and menus, using appropriate HTML elements and ARIA attributes.
search, compare, book, buy, request a quote and more.
to identify points of friction.
Is your website ready to be understood and used by AI agents?
Discover our Agent Readiness Audit