How Zurich Insurance UK monitors AI search with Oncrawl

As AI-powered search gained prominence and began reshaping how users find information, Zurich Insurance UK recognized the need to understand AI bot behavior on their website. The Search Marketing team wanted visibility into how platforms like ChatGPT, Perplexity and Google’s AI Overviews were interacting with Zurich’s insurance content.

Take a look at how Zurich Insurance UK was able to utilize Oncrawl to solve their problem.

Challenges

The Zurich team faced several challenges regarding AI search. Traditional analytics didn’t reveal:

• AI bot access: Distinguishing AI bots from regular traffic was impossible, leaving Zurich unable to see how these bots interacted with their site.
• Citation accuracy: When AI tools reference insurance information, accuracy is paramount. The team needed to identify which content AI bots accessed to verify citations were correct.
• Resource utilization: Without visibility into which resources ChatGPT and other AI bots were crawling, opportunities to optimize or restrict access accordingly went unnoticed.
• Different bot purposes: Not all AI bots serve the same purpose. The team wanted to distinguish between training bots, search bots, and user-initiated requests to optimize appropriately.

Solution

Zurich began using Oncrawl’s AI bot monitoring capabilities to answer their questions. Through log file analysis, they gained comprehensive visibility into AI search engine behavior and tracked multiple AI bots across their logs.

Thanks to Oncrawl’s dedicated dashboard, they saw the three distinct categories of AI bot activity:

• Training bots: for model development
• Search bots: for asynchronous indexing
• User bots: for real-time queries

Sample dashboard

The dashboard also showed which AI bots accessed specific content, including PDFs and resources that wouldn’t appear in traditional analytics.

Additionally, the team used Oncrawl to cross-reference AI bot activity with zero-click searches and impressions, identifying content used to generate AI-powered answers and prioritizing optimization on high-value pages.

Results

“Oncrawl has been invaluable in providing additional technical and content insights. The AI search reports, alerts and monitoring saves a lot of time. This allows me to flag examples that may be missing from traditional analytics and recommend technical or content improvements.”