Troubleshooting Lost Links
Frequently Asked Questions
What's Covered?
In this guide you’ll learn more about how to investigate links marked as lost in the Moz Link Index and Link Explorer.
Quick Links
Troubleshooting Links Marked As Lost In Moz’s Index
When investigating your lost links in Link Explorer, it’s important to remember that there are lots of reasons a link may be marked as lost, only one of which is that the link was removed or updated. Below, we’ll outline some of the reasons a link may be marked as lost to help you get a handle on recovering those links in your backlink profile.
How To See Your Lost Links
Before you can start investigating your lost links, you need to know how to see which links have been identified as lost in our index.
There are 2 ways to see links marked as lost in the index.
1. Head to Inbound Links and filter by Link State. This will show you all the inbound links for the site you’re researching which are marked as lost. You can then export this data to .csv or you can add these links to a Link Tracking List to keep track of.
2. Head to the Discovered and Lost section in Link Explorer and toggle the Display option to Lost. This will display all the domains which have been marked as lost. You can export this information to .csv or add these domains to a Link Tracking List to keep track of. Please note: If a link still exists from this domain to the site you’re researching, the domain will not be marked as lost. Lost linking domains will only include domains which no longer have any links flagged as active pointing to the site.
Date Lost
Date Lost is the date our crawler attempted to recrawl the link noted and wasn’t able to. It doesn’t necessarily mean the link has been removed, rather when our crawler attempted to access it, it wasn’t able to. This could be because of something like a server error, the link being moved or updated, or the page containing the link returned a 404. If you know that the link still exists you can keep track of this link by adding it to a Link Tracking List - that way you'll know when the link is rediscovered.
So why are links marked as lost? Check out some of the reasons below to get to the bottom of why your links may be missing from our index.
The Link Was Removed, Updated, or Moved
One of the most common reasons a link is marked as lost is because the link was removed, updated, or moved. If our crawler attempts to recrawl a link in our index and the link is no longer there (or it’s different), the original link will be marked as lost in our index.
You can verify if a link has been changed or removed by heading to the source code of the referring page and searching for your link. To access the source code, right click in the browser window and click View Source.
If you search the source code of the referring page and your link is still present, but it’s marked as lost in our index there are various other reasons this could have happened which we will explore below.
Noindex Tags Have Been Updated
While you are looking at the source code of the referring page, be sure to check to make sure there isn’t a noindex tag on the link itself or on the page. If the page containing a backlink to your site is marked with a meta robots tag of noindex, nofollow or noindex, follow, our crawler will not be able to index the links on that page. This means for pages marked as noindex, any links on that page will not be added to the index and any specific links marked this way on a page will not be indexed, either.
If a link was previously able to be indexed but the noindex tag is updated, the link will then be marked as lost in our index the next time we try to crawl it.
You can check the page’s meta index directives in the source code. To access the source code, right click in the browser window and click View Source.
Server Blocks and Broken Pages
Sometimes when our crawler heads back out to recrawl a link, the server hosting the referring page returns a status code which doesn’t allow us to continue to crawl. If, at the time we attempt to recrawl the page your link is on, the server returned an error or blocked us we won’t be able to recrawl your link and it will be marked as lost in the index.
Status codes which can inhibit our ability to crawl are 4xx and 5xx status codes. These come directly from the server of the site and tell our crawler it’s unable to access that page. Similarly, if a referring page has been removed or is now returning a 404 error, our crawler will mark links it had previously found on that page as lost.
Sometimes these issues are intermittent (like 5xx errors and temporary outages). If they are, the next time we attempt to recrawl those links and are able to access them, they will be marked as active in our index.
We suggest adding links to a Link Tracking List to keep track of when we recrawl them.
Updates to Robots.txt or Nofollow Tags
Similar to how server blocks and broken pages can affect our ability to index a particular page and its links, updates to robots.txt files or nofollow tags can also impact our crawlers ability to move forward.
If a site has updated their robots.txt file to block our crawler, we will not be able to crawl the links on that site and follow them to the sites they are linking to. This can inhibit our ability to recrawl your links and they will be marked as lost. For example, if we had crawled Site A to Site B to your site but now Site A is blocking our crawler, dotbot won’t be able to crawl Site A to find the link to Site B and so on.
If a site has implemented nofollow tags on links to the referring pages for your site’s backlinks, our crawler will not be able to follow those links to crawl the referring pages. This will cause your links to be marked as lost in the index unless we are able to find a different way to access that referring page.
What’s Next?
If you find that any of these issues applies to your link or the page your link is on, these would need to be updated and changed on the site that’s linking to you. Unfortunately, Moz does not have the ability to make any changes on those sites and would not be able to resolve this issue on your behalf. This includes our crawler being blocked by the server (as mentioned in the server blocks section). The response we are seeing in this case comes directly from the server of the site and would need to be explored with the site admin or hosting provider to resolve.
We also recommend adding lost links you’re investigating to Link Tracking Lists in Link Explorer. This can help you to keep an eye on those links and see when we reindex them or mark them as active again in our index.
If you need further assistance investigating a link which is marked as lost, our Help Team will need the follow information in order to best help you:
- The URL for the referring page where the active link is located
- The site you’re researching in Link Explorer which shows the link as lost
- If possible, a screenshot of Link Explorer
Please email as much of this information as possible to help@moz.proxybugs.com.
Related Articles
Was this article helpful?
Yes! Amazing!
Yes! It was what I needed.
Meh. It wasn’t really what I was looking for.
No, it wasn’t helpful at all.
Thanks for the feedback.