Find news to use on May 12, 2026
Here are the topics I’ve been discussing with my clients lately.
1. An increase in crawled – currently unindexed – pages
There has been a lot of discussion on social media about Google showing more pages marked as “crawled – not currently indexed” in the Page Indexing section of GSC. These are pages that Google has crawled and does not want to index.
Almost every website I monitor has seen an increase in crawled, currently unindexed pages since March 2026.
Note: I put these images in a folder and asked Antigravity to combine them into one image for me. It has Python written to do this. 🦾
I would strongly recommend that you take some time to read this document from Microsoft Bing the evolving role of the index: from ranking pages to supporting answers.
They say: “Search indexing is designed to help people decide what to read. Informed indexing is designed to help AI systems decide what to say.”
I can’t prove this, but I believe Google is becoming more selective with indexing as the index increasingly serves as the basis for AI. There are still some sites that people want to visit, but if you’re creating content that many others have already written about, there’s limited need for it to be in the index.
Check whether your crawled pages that are not currently indexed are standard content
However, I found it interesting that at the Google Search Central event I attended in Toronto they talked about crawled, currently unindexed pages. They said crawled pages – not currently indexed – could be due to technical issues, or “we looked and found it wasn’t good.”
There was also a lot of discussion about commodity content at this event.
I took this photo. I feel like Danny is judging me. 😂
I would encourage you to try this exercise.
Go to GSC → Indexing → Pages and click “Crawled – not currently indexed”
It makes sense that some of these pages are not included in the index, especially if they are feed pages, page breaks, or non-canonical variations such as pages with or without a trailing slash in the URL.
Find some URLs that you really want in the index. Open it in Chrome, open Gemini in the sidebar and use this prompt:
Is this product content? Is it likely that many others on the Internet have a page with similar content?
You can then ask Gemini for ideas on how to improve it so that it is uniquely valuable and index-worthy.
2. Preparing for the Agent Web
Google CEO Sundar Pichai recently stated, “Search will be an agent manager where you do a lot of things.”
“The Agent Network” is an Internet in which autonomous AI agents, instead of humans, navigate websites, use tools, and communicate with other agents to help people achieve goals.
What does this mean for your website?
Try this prompt in your favorite LLM:
I read that the web is becoming an agent. Help me understand what this means for my business. My website is (URL).
Our role as SEOs will shift from the task of getting displayed by search engines and instead ensuring that AI agents choose our company to help their user complete a task.
We also need to help figure out how to convert these agent visits into business opportunities. I think there are so many possibilities here. At the same time, there is no playbook for us to follow.
This week Google released an important read We make our websites agent friendly. Agents currently have three ways to rate web pages: via screenshots, via the HTML code on the page, and using something called Accessibility tree. The accessibility tree is essentially a view of what is important on a page, especially when it comes to taking action. It’s designed for tools like screen readers, but it also helps agents understand pages.
I used Antigravity to build a tool that allows you to evaluate a page for anything that might prevent agents from seeing or using your site, which was recommended in Google’s blog post:
Test your website with my agent readiness tool.
Once you receive your results, a prompt will be issued that you can use in your preferred LLM to determine whether the issues raised are worth working on.
Testing Agent Claude on customer sites
In one of my client calls recently, we opened up Claude in the Chrome extension This will move it to the Chrome sidebar. Claude can currently perform agent actions on one side. Gemini can’t do that, but I expect we’ll be hearing more about Gemini taking action soon.
This page has a form that submits information through an API and then provides a response and a link to purchase, which is how this company monetizes their website. We asked Claude for an offer. It asked us some questions that were required in the form. It didn’t type on the screen, but instead figured out what it needed to do to connect to the API. We received a response and the purchase link very quickly.
The customer has now worked on implementing WebMCP for this site. This makes navigating and using browser-based agents even easier. (Here’s an excellent guide too Implementation of WebMCP edited by Suganthan Mohanadasan.)
On another customer call, we used Claude in the browser to help us select a product and then add it to our cart. Claude realized we had many options to choose from and shared them with us. In this case, Claude failed. I asked, “Were there any technical issues with this website that could be improved to help you accomplish this task better?” Claude provided us with an extensive list of improvements, including improving form dropdowns, fixing a broken link on the page, and improving some other JavaScript-driven actions that were difficult for the AI to perform.
This site is not currently committed to implementing WebMCP, but has committed to it UCP waiting list as UCP will probably be more important to them than WebMCP.
I assume, but I don’t know for sure, that sites that are easy for agents to use to help achieve goals will be in higher demand among those agents. However, we don’t know exactly how this will happen. Will there be a directory of WebMCP enabled sites? It makes sense to me that this might just be part of the information that Google stores on a page when it is crawled – regardless of whether it is agent friendly. We’ll see!
3. Learn to use Google antigravity
I know everyone is talking about using Claude Code, Claude Cowork or OpenAIs Codex right now. I still really, really love Antigravity. I use it all day long.
I don’t know for sure, but I could imagine a world where we have a connection to Antigravity through the Chrome sidebar and other places where we can use Gemini. I think it’s possible that antigravity and Gemini are deeply embedded in everything we do.
Here’s what I did with Antigravity this week:
Gave him a bunch of screenshots and asked him to combine them into one image For me.
Developed a tool that can be connected to the GA4 MCP and informs me daily about which pages of my website people are engaging with. It opens a browser and reads the comments and reactions to posts in the Search bar. We then summarize what users have found helpful when using Gemini 3 Flash. Over time, I hope to add agent workflows that analyze what my audience finds helpful and use that as a guide to better support search bar members.
I developed a system that adds stories that I have marked as important to a Google Doc along with tags for the customers I want to share these stories with. Now I can say to Antigravity, “Create the monthly newsletter for (client name)” and it will create me a Google Doc and remove the tags from the master document.
I used a workflow I created last month to perform searches in AI mode and capture screenshots for a customer report. Gemini 3 Flash is then used to analyze what has changed in terms of my client’s SERP features or presence.
I used a workflow I created last month to analyze competitor websites and archive.org versions and report on what changes they have made and what new blog posts they have written.
Email triage via Google Workspace CLI. This is so good.
LLM recommendations data retrieved from the GA4 MCP and pasted it directly into my client’s Google Doc. This part of my reports now takes me less than 5 seconds.
I built it Agent readiness toolcreated and deployed to Google Cloud for everyone to use.
I ended that newsletter episode and said to Antigravity, “Put this on my WordPress site.” And it used a workflow we created that connected to my WP API, optimized the images, and created a perfect draft for me to approve.
I’ve been playing with using my webcam to create an HR and HRV sensor This uses the color in your face. It’s so damn cool. I’m trying to create a tool to help people refine their bodies for more joy. We’ll see if it works!
That’s fair some the work I did with Antigravity. I would be here all day if I included everything. I love it so much.
I found a few other stories interesting
Google has announced updates for AI mode including excerpts from online discussions and additional links to websites.
UCP was discovered in traditional search results
Gemini Intelligence is coming to Android – It can automate tasks for you, like booking things.
Data from Shopify says AI-referred shoppers convert 50% better and spend 14% more
Papa John’s is now offering drone deliveries in collaboration with Google’s Wing
Robots are coming that can do housework. Sorry for the shouting. I can’t wait for robots to do the housework for me.
That’s all for this week. But so much more happened. If you miss my full report on everything This is what happens, you can find it here in the Search and AI newsfeed in my community the search bar.
Sign up for the newsletter
If you would like to be notified by email every time I publish a newsletter like this in the future, you can sign up here:
Or, Join us in the search bar for the latest news on search and AI. Or better yet, join my paid community where we call regularly to brainstorm on the use of AI and I also share tips and ideas.
By the way… Antigravity helped me create code snippets to dynamically insert the best conversations in the search bar into my blog posts. 🦾