Hard days: Google dropping pages from index
Submitted by webaurora on Mon, 03/19/2007 - 19:08.
"Google hates me"... That joke turns out to be not especially funny when your own pages are being dropped from Google index :x Yesterday Google started to kick out my pages from index. Gradually, one by one. One per day. My website is 5 and a half months old, but it was already pretty well ranked in Google (after initial well ranking, dropping and babbling up again) and got quite a lot of traffic from natural search. One of the pages dropped yesterday is second most popular page of the website. The only changes I have done to the website recently is uploading a few new pages with mostly affiliate links (there are about 8 pages with pure unique content that are are not dropped yet on my website). They are not indexed yet, but the pages thrown away by Google are like that - product items with affiliate links. Any ideas what I displeased Google with? Is it annoyed with affiliate links and doesn't want those pages in index? Anything else? What to do?
Any advice is highly appreciated. Thanks in advance.
Hard days: Google dropping pages from index
Hi WebAurora,
Have you checked the status of your site on Google's Webmaster Central yet? I've found it's a great resource for troubleshooting indexing, crawling and other SEO issues:
http://www.google.com/webmasters/
Check out the blog too:
http://googlewebmastercentral.blogspot.com/
Best,
Jeremy
Hard days: Google dropping pages from index
Hi Jeremy!
Thanks a lot for the valuable links! The webmaster tools are great indeed, that was extremely helpful, thanks again.
At the first step I was advised that pages from my site are included in Google's index and Googlebot last successfully accessed my home page on Mar 18, 2007. However, it was a remark about potential indexing problems: "We do not know about all the pages of your site. You can submit a Sitemap to tell us more about your site. "
Actually, I havent submitted a sitemap to Google before because on the one hand, the site is not completely ready yet and I planned to do it when all pages are up, on the other hand, all pages uploaded on the server were fast and successfully indexed. It seems strange to me that if pages were in Google index for a couple of months, they can be dropped just because of abscence of a sitemap, when you even don't change internal linking structure.
Anyway I submitted a sitemap to Google, and so far I can't see any problems with crawling or indexing in detailed diagnostic and statistics within my webmaster tools. Sitemap status is also OK, web crawl doesn't report ANY mistakes. But the fact is that Google thoroughly picked out and dropped all product pages with mostly affiliate links from index and left only 12 ones with unique content (where 1-5 affiliate links are embedded).
I guess I should wait a couple of days, when Google crawls my website again and refreshes its database before making conclusions.
So cross my fingers and wait... :wink:
Hard days: Google dropping pages from index
One thing to keep in mind is that your site will naturally fluctuate in the rankings when it's new. It's unusual for Google to drop all of your pages, but it sounds like you diagnosed some of the problems.
Do you think it's possible that some of your pages fell into supplemental results:
http://www.jimboykin.com/damned-to-google-hell-supplemental-results/
Hard days: Google dropping pages from index
I also find that sometimes Google removes my sites from the index from a couple of days, however they will be back later.
This new algorythm is incredible. You need to go with LSI. I use Kwbrowse.com, it's free and highly effective.
Also, don't go to broad in optimization.
Hard days: Google dropping pages from index
Thanks a lot, Jeremy, for your post and right dignosis for my website. After the shocking revelation yesterday about sending most of my pages from main to supplemental index, reading about gloomy prospects that "once pages are sent to the Supplemental Results, they tend not to get revisited", I analysed every fact thoroughly, went through everything I could find about supplemental results and finally made some conclusions why this could happen, but today Google struck me again... It's just no logic in what is happening!
Well, step by step. Yes, I have experienced fluctuation in rankings for new sites, but up to now my pages have never been thrown out from Google index completely without any obvious reason. I checked the domain yesterday, and, oh no, all pages except home suddenly became supplemental results! Fact no. 1: still a couple of weeks ago, I know for sure, they (or at least most of them) were in main index. The pages have mostly unique content, not orphans of course. So why were they moved?
Fact no. 2: Product pages that were dropped by Google completely, were originally in main index and then disseapered. They didn't go to supplemental results. Well, it's hardly possible to have unique content in pages describing very similar products, so I can assume they were recognised by Google as duplicate content and thrown away from index.
Fact no. 3: Early indexed product pages were dropped by Google from index in 1-2 day after I put up some new similar product pages. Here I could think the following assumption from seobook.com http://www.seobook.com/archives/001545.shtml could be applicable: "If too much of your site is considered to be useless or duplicate junk then Google may start trusting other portions of your site less." OK, let's assume, Google didn't like new product pages looking similar to each other and other pages on the web, stopped trusting my website and put nearly all pages in supplemental results. It seems to be at least logical so far.
Fact no. 4: Today Google indexed one of recently uploaded product pages and put it in main index! And in a couple of hours after that, one of product pages dropped from index before appeared their again! In main index. These 2 pages are actually least worth to be in main index, especially as compared to the ones with content that are currently in supplemental results.
I don't know to what extent we can trust this source http://www.seo4fun.com/notes/supplementals.html, but idea that "Previously indexed pages are dropped, leaving only supplemental copy behind. In this case, your page didn't "go supplemental" due to any obvious duplicate content reasons. Instead, your problem could be due to lack of PageRank or a bad outbound link to a crap site that reduces your page's trust score. A few high quality inbound link may set things straight. " sounds reasonable in this case. Anyway, more optimistic :wink:
Or on the contrary I should prepare that when product pages are up as long as content pages were before they went supplemental, they will suffer they same fate? :x
Thanks in advance for your opinions.
Hard days: Google dropping pages from index
Just a quick update to my last post. By now Google released most of pages from supplemental results and put them back to main index. Mystery... :o