AI Security · Inbound Agent Traffic

AI Agents Are Visiting Your Website. How Should You Govern Them?

By Infonaligy · Updated November 16, 2026 · 9 min read

INBOUND AGENT TRAFFIC // POLICY

Your next buyer may never see your homepage. Their AI assistant will. Personal agents, agentic browsers and procurement bots are already reading pricing pages, filling quote forms and opening support tickets on behalf of real people. That traffic is not a bot problem or a marketing opportunity. It is both, and IT Directors and CXOs need one policy that treats it that way.

What changed in inbound AI agent traffic this year?

Inbound AI traffic stopped being one category in 2026, and the tooling finally caught up. Cloudflare said on July 1, 2026 that it was splitting AI traffic into three categories, Search, Agent and Training, so site owners could treat each differently. From September 15, 2026, Cloudflare said new domains would get defaults that block Training and Agent bots on pages with ads while still allowing Search.

Then, on November 4, 2026, Cloudflare announced per-agent analytics and access controls for personal AI agents. Based on secondary coverage of the announcement, it appears to have extended its Agents policy (renamed "AI Crawler and Agent access policies") so owners can set different terms for each supported personal agent, and it added a Personal Assistants category to bot analysis in Security Analytics.

Industry coverage this fall has also reported that agentic AI traffic is growing quickly. The exact figures vary by source and by site, so measure your own. The direction is not in question: a growing share of your visitors are software acting for a human.

How should you classify inbound AI agent traffic?

Classify inbound automated traffic into four buckets, because each one deserves a different answer.

  • Search crawlers. Index your content so AI search and answer engines can cite you. Usually welcome on public pages.
  • Training crawlers. Collect content to train models. No direct visit, no referral, no lead. A business decision, often restricted.
  • User-directed agents. A personal assistant or agentic browser acting for a specific person right now: comparing vendors, requesting a quote, checking an order. This is a customer or prospect, just not in a browser tab.
  • Malicious or abusive bots. Credential stuffing, pricing scrapers, fake form fills, inventory hoarding. Some now dress up as AI agents because "agent" sounds legitimate.

The hard part is telling bucket three from bucket four. That is an identity problem, not a content problem.

How do you verify that an agent is who it claims to be?

Do not trust the user-agent string; verify cryptographically wherever you can. A user-agent header is a label anyone can type. Scrapers routinely claim to be well-known crawlers or popular assistants.

The better pattern is signed agent identification. Web Bot Auth, built on HTTP message signatures, lets an agent operator sign requests with a published key so your edge can confirm the request really came from that operator. Pair it with what you already have:

  • Signature or verified-bot status first. If your CDN or WAF reports a verified agent, that beats any header.
  • IP and ASN checks against the operator's published ranges as a secondary signal.
  • Behavioral signals such as request cadence, navigation patterns and form timing.
  • User authentication for anything personal. Verifying the agent tells you which software is calling. It does not tell you which customer authorized it. Logged-in actions still need the human's session, scoped delegation or step-up confirmation.

Treat unverified traffic that claims to be an agent as anonymous at best and hostile at worst. This is the inbound mirror of the zero trust model for AI agents: identity first, least privilege, continuous verification.

Which surfaces should agents be allowed to touch?

Decide per surface, not per site. A blanket "block AI" rule hides you from buyers' shortlists, and a blanket "allow" invites abuse on your most sensitive flows. Use a matrix like this as a starting point.

SurfaceSearch crawlersTraining crawlersVerified user agentsUnverified / suspiciousHand to a human when
Marketing and blog pagesAllowBusiness decision (often restrict)AllowRate limitRarely needed
Product and pricing pagesAllowRestrictAllow, rate limitedChallenge or block bulk accessCustom or negotiated pricing
Quote and contact formsNo accessNo accessAllow with validationChallengeLarge deals, unusual terms, regulated buyers
Support and knowledge baseAllow public articlesRestrictAllow, ticket creation with confirmationRate limitComplaints, escalations, safety issues
Logged-in customer portalNo accessNo accessRead-only with user sessionBlockAnything beyond viewing
Checkout and ordersNo accessNo accessAllow with step-up confirmationBlockHigh value or first-time buyer
Account changes (email, bank, MFA, users)No accessNo accessHuman onlyBlock and alertAlways

The right-hand column matters most. Agents can gather, compare and draft. Changes to payment details, contact info, authentication or user access should always land with a human through a channel the agent does not control.

How should rate limits and abuse controls change for agents?

Set rate limits per verified agent and per surface, and keep them tight where money or credentials are involved. A personal assistant comparing three vendors needs a handful of page loads, not thousands.

Watch for the abuse patterns agent traffic makes easier:

  • Credential stuffing disguised as "an assistant logging in for the user." Enforce MFA, bot scoring and lockouts on login regardless of claimed identity.
  • Pricing and catalog scraping by competitors. Publish the pricing you want agents to see, and rate limit the rest.
  • Fake or junk form fills that pollute your CRM and waste sales time. Validate fields, verify email domains and score submissions before routing.
  • Support flooding through automated ticket creation. Cap tickets per account and per agent.

How do you make your site readable for the good agents?

Make your important pages easy for machines to read accurately, because agents increasingly build the vendor shortlist before a human visits. If an assistant cannot find your services, service area or pricing model, it will recommend a competitor it can read.

  • Structured data (schema.org Organization, Service, Product, FAQ) so agents extract facts instead of guessing.
  • Clean product and pricing pages with real text, not prices locked inside images or scripts.
  • An llms.txt file pointing agents to your most authoritative pages. Treat it as a helpful map, not a security control.
  • Agent-friendly forms with clear labels, predictable fields and explicit confirmation steps.
  • Consistent facts across your site, profiles and listings, so agents do not have to reconcile conflicting answers.

This connects directly to how buying is changing. Our look at agentic AI in B2B sales covers the seller side; here the point is simply that being legible to agents is now part of being findable.

What should you log and measure?

Log agent traffic as its own segment so you can see what it costs and what it earns. Most analytics tools still blend it into "bots" or drop it entirely.

  • Requests by category (search, training, user agent, unverified) and by surface.
  • Verified agent identity, verification method and outcome (allowed, challenged, blocked).
  • Form submissions and tickets that came through agents, and how many turned into qualified opportunities.
  • Blocked attempts on login, checkout and account changes, routed to your security monitoring.

Edge analytics, such as the Personal Assistants view Cloudflare reportedly added, help, but join them with CRM and support data so you can answer the business question: are agent-sourced leads real? Security vendors are also building agent observability (CrowdStrike, for example, announced plans to acquire Fiddler for AI agent observability, according to November 2026 reports), though most of that work targets agents you run, not agents visiting you.

Who owns the inbound agent policy?

Security owns the controls, marketing owns visibility, sales owns the handoff, and one executive owns the tradeoffs. Without a named owner, marketing will open everything and security will block everything.

  • Security / IT: verification, WAF and bot rules, rate limits, logging, incident response.
  • Marketing: which pages are agent-readable, structured data, llms.txt, training crawler stance.
  • Sales and customer success: what agents may submit, and how agent-originated quotes and tickets get routed and qualified.
  • Legal / finance: terms of use for automated access, pricing exposure, payment and refund rules.

Note the contrast with outbound policy. Our guide to agentic browsing controls covers your employees' agents browsing the web. This policy covers everyone else's agents browsing you. You need both.

What does a 30/60/90-day plan look like?

Days 1–30: See it

  1. Turn on AI and bot categorization at your CDN or WAF and review current defaults, especially on domains added after September 15, 2026.
  2. Inventory surfaces: public, pricing, forms, support, portal, checkout, account changes.
  3. Baseline traffic by category and surface. Flag anything claiming an agent identity without verification.
  4. Name an executive owner and a cross-functional working group.

Days 31–60: Decide it

  1. Approve a surface-by-surface matrix like the one above.
  2. Prefer signed or verified agent identification; challenge or block unverified agent claims on sensitive surfaces.
  3. Set per-agent and per-surface rate limits. Harden login, checkout and account-change flows with MFA and step-up confirmation.
  4. Define human handoff rules and wire them into CRM and ticketing.

Days 61–90: Earn from it

  1. Add structured data, clean up pricing and service pages, and publish an llms.txt.
  2. Tag agent-originated leads and tickets, and report conversion and abuse monthly.
  3. Run a tabletop exercise: a spoofed "assistant" attempts credential stuffing and a bank-detail change.
  4. Review the policy quarterly as edge providers add per-agent controls.

If you want help, Infonaligy's AI security team builds and runs these controls, our AI DevOps practice wires verification and logging into your edge and apps, and CRM and sales AI work makes sure agent-sourced leads land cleanly. For the policy itself, start with AI consulting.

The bottom line

AI agents are now visitors, buyers and attackers all at once. Classify them, verify them cryptographically instead of trusting headers, decide access surface by surface, keep account changes and payments with humans, and make your public pages easy for good agents to read. One named owner, one matrix, one 90-day plan.

Frequently asked questions

What is inbound AI agent traffic?

It is automated traffic from AI systems visiting your website, portal, forms or support flows. It includes search crawlers, training crawlers, personal assistants and agentic browsers acting for a real person, and malicious bots posing as agents.

Should we block AI agents from our website?

Not across the board. Blocking everything can keep you off the shortlists that buyers' assistants build. Allow verified search and user agents on public pages, restrict training crawlers as a business decision, and keep logins, checkout and account changes tightly controlled.

Can we trust the user-agent string to identify an AI agent?

No. User-agent headers are easy to spoof. Prefer signed agent identification such as Web Bot Auth with HTTP message signatures, verified-bot status from your CDN or WAF, and published IP ranges as a secondary check.

What should AI agents never be allowed to do on our site?

Agents should not change account email, bank or payment details, MFA settings or user access. Those actions should always require the human, confirmed through a channel the agent does not control.

Who should own our AI agent access policy?

Name one executive owner. Security runs verification, rate limits and logging, marketing decides what is agent-readable, and sales and support define how agent-originated leads and tickets are handled.

Infonaligy delivers AI security and agent governance remotely nationwide from our home base in Dallas–Fort Worth; see all locations we serve on site.

Inbound Agent Governance

Ready to set rules for the AI agents visiting your business?

We will map your surfaces, review your edge settings and agent traffic, and deliver a decision matrix and 90-day plan your security, marketing and sales teams can all sign off on.

DFW home base · remote nationwide · governed by default · 800-985-1365