If you have the budget to purchase AI visibility tools like Profound or Scrunch AI, this article may not be for you.
This guide is for SEO professionals and business owners who want to measure AI search performance without paying for premium tools.
You can start with five data sources:
- Google Search Console — AI Performance
- Bing Webmaster Tools — AI Performance
- Server log files
- Google Analytics 4 (GA4)
- Microsoft Clarity
The goal is simple: understand whether AI bots are accessing your pages, whether your content is appearing in AI search, and whether visitors are coming to your website from AI platforms.
Step 1: Select Your Important Pages
Don't track every URL on your website. It creates too much data and makes analysis difficult.
Start with your most important pages:
- Product and service pages
- Comparison pages
- High-intent landing pages
- Blogs that target important business questions
Use the same list of URLs throughout your analysis.
Step 2: Collect Google Search Console Data
Open Google Search Console and look for the Generative AI performance report, if available for your website.
Export the available data for your selected period and record AI impressions by page and week.
This helps you understand how your pages perform in Google's supported AI search experiences.
Step 3: Collect Bing Webmaster Tools Data
Open Bing Webmaster Tools and access the AI Performance report.
Export the available citation data and organize it by page and week.
This helps you understand which pages are cited in supported AI experiences, including Microsoft Copilot.
Note: Google AI impressions and Bing citations are different metrics. Keep them separate.
Step 4: Analyze Server Log Files
Download your server or CDN logs for the same period.
Identify relevant AI search and retrieval bots, then group their requests by page and week.
For example, OpenAI uses different crawlers for different purposes:
- OAI-SearchBot helps discover content for ChatGPT search.
- GPTBot is associated with crawling for potential model training.
- ChatGPT-User is used for certain user-triggered fetches.
If your goal is to measure AI search activity, focus on the relevant search and retrieval bots. Keep training-related crawling and user-triggered fetches separate.
Also, separate successful page responses from failed requests and remove irrelevant URLs.
Your output should show how many times relevant AI crawlers requested each important page during each week.
Remember: A bot request means that a page was fetched. It does not mean the page was cited or shown to a user.
Step 5: Collect AI Referral Traffic From GA4
Open GA4 and go to Reports → Acquisition → Traffic acquisition.
Identify traffic from AI platforms such as ChatGPT and Perplexity, along with other sources where referral information is available.
Record the following by landing page and week:
- Sessions
- Engaged sessions
- Conversions
This helps you understand whether AI platforms are sending visitors to your website.
Some AI-assisted visits may appear as direct traffic, so GA4 will not capture every visit originating from an AI interaction.
Step 6: Use Microsoft Clarity to Understand Visitor Behaviour
Use Microsoft Clarity to analyze visitors who arrive through identifiable AI referrals.
Check which pages they visit, how far they scroll and whether they interact with important elements such as enquiry forms.
This helps you understand what visitors do after reaching your website.
Clarity measures visitor behaviour, not AI citations.
Step 7: Combine the Data in One Spreadsheet
Choose a fixed reporting period, such as three months, and divide it into weekly intervals.
Create a spreadsheet with these columns:
- Page URL
- Week
- AI bot requests
- Google AI impressions
- Bing citations
- AI referral sessions
| Page URL | Week | AI bot requests | Google AI impressions | Bing citations | AI referral sessions |
|---|---|---|---|---|---|
| /seo-services/ | Week 1 | 35 | 120 | 8 | 12 |
| /seo-services/ | Week 2 | 48 | 150 | 12 | 18 |
| /seo-audit/ | Week 1 | 5 | 40 | 1 | 3 |
The figures above are examples, not actual performance data.
Keep the metrics separate. They measure different stages of AI search performance and should not be added together into one score.
Step 8: Understand What the Data Is Telling You
Before comparing the numbers, you need to understand what each data source means.
Each tool tells you something different about your website.
- Server log files = Crawled: An AI bot visited and requested your page. This does not mean your page was cited.
- Google Search Console AI Performance = AI visibility: Your page received impressions in Google's reported AI search experiences.
- Bing Webmaster Tools AI Performance = Citations: Your page was cited in the AI experiences covered by Bing's report.
- GA4 = Clicks and visits: People clicked through from an AI platform and visited your website, where the referral could be identified.
- Microsoft Clarity = Visitor behaviour: You can see how visitors interact with your website after they arrive.
In simple terms:
Crawled → Visible or cited → Clicked → Visited your website
These are different signals. One does not guarantee the next.
How to use these signals
| Crawled? | Visible or cited? | What does it mean? | What should you do? |
|---|---|---|---|
| High | High | AI bots are requesting your pages, and your content is receiving visibility or citations in the reports. | Maintain the pages, update the content and improve internal links. |
| High | Low | AI bots are visiting your pages, but reported AI visibility is low. | Review whether your content answers user questions clearly and offers useful, original information. |
| Low | High | Your pages are receiving reported visibility despite few recent bot requests in your logs. | Check the reporting dates, crawler coverage and whether cached content or earlier crawling explains the result. |
| Low | Low | You have little recorded crawler activity and little reported AI visibility. | Check robots.txt, page access, rendering, indexability and internal links. |
High and low are relative to your own website and the period you are measuring. There is no universal number that defines high AI visibility.
Important: Google AI impressions and Bing citations are not the same metric. Keep them separate in your spreadsheet. Use each report to understand the AI visibility it actually measures.
Also, do not assume that more bot requests automatically mean more citations. The purpose of this analysis is to identify patterns and understand which pages need improvement.
Step 9: Track Performance for Three Months
Repeat the same process every week for approximately three months.
Compare changes in:
- AI bot activity
- Google AI impressions
- Bing citations
- AI referral sessions
- Conversions from identifiable AI referrals
Record when you make important technical or content changes.
If visibility improves consistently after an update, investigate whether the change contributed to the improvement. Other factors, including platform updates and competitor activity, may also affect results.
Final Thoughts
You don't need premium AI visibility tools to start measuring your website's AI search performance.
Use Google Search Console, Bing Webmaster Tools, server logs, GA4 and Microsoft Clarity to understand crawler access, AI visibility and visitor behaviour.
The goal is to connect these signals and identify which pages need attention.
When you identify low visibility pages, discover what your buyers actually ask using our guide on How to Do Keyword Research for AI Visibility.
If you need help identifying technical issues, content gaps and opportunities to improve AI search visibility, contact me for an AI visibility audit.