Sunday, October 12th, 2008
This is a follow up post to my previous posts about my friend’s Google ranking drop. As you may remember, his Google ranking was restored a few weeks after he blocked the proxy website from copying his entire website and submitted a Google reinclusion request. As you may have guessed, he was quite thrilled to see his SERP ranking shoot up again.
Well, as luck would have it, I received a phone call last night from my friend telling me that his website was bombing again. I Googled his favorite keywords and they seemed to rank fine over at my end, but he explained that he traffic stats from Google was flat. They nosedived a day or two ago. I chalked up the results I was getting to Google adjusting the results.
This new twist got me thinking. What in the world could be making this website’s ranking bounce around like this? Looking back, the proxy website may not have been 100% at fault. There has to be something else.
I began doing a little research and learned about few things about duplicate content. The reason I looked at that particular area is because there is absolutely nothing else I can find wrong with this website. Duplicate content seems to be a rather popular culprit.
I came across a pretty well laid out website called “Google Rankings Diagnostics” that describes a whole heck of a lot of issues you might be having with your website. This website validated what I pretty much already knew…that if you have multiple URLs (on a domain) with the same exact content, Google has trouble figuring out which page is the original and may throw all of them out.
I took a very close look at my friend’s website. Again, I took a unique line of text from his homepage and searched for it in Google (inside quotes). A funny thing happened. I saw the homepage result, but there were a few extra results as well, all on his domain. There were about 5 extra pages in total.
Now, some of these extra results have been there for years, so I don’t attribute the issue to those pages being duplicate content. What struck me was one of the extra pages.
A few months ago, my friend moved one of his pages. He put a 301 redirect in his .htaccess file, which was the correct thing to do. So now, the old directory where the page was held forwarded to a new page. It looked something like this:
Redirect 301 /olddirectory/ http://www.hiswebsite.com/newpage.php
The redirect worked fine, but here is what that extra page in the search results looked like:
http://www.hiswebsite.com/newpage.phpoldpage.php
Guess what page was showing at that URL…yup, the homepage. The dynamic nature of his website sends unknown page results like this to the homepage. This was a fluke. My friend forgot that there were pages inside the old directory he redirected to the new page. Every old page in that old directory was tacked on to the new page, like you see above. To make matters worse, there were a bunch of links from other websites pointing to the old pages in the old directory.
I am not sure if this would cause the ranking drops that he is experiencing, but the timing certainly lines up with when the issue began. It is also certainly considered duplicate content.
So, here is what I did to deal with the issue this time. I deleted the redirects in the .htaccess file and blocked the URLs of all those extra results in the robots.txt file. Hopefully, this will tell Google to not spider or index those pages and it will also tell Google that those links into the site are dead.
Now, we have to wait. I am not going to submit another reinclusion request to Google because I want to see if the ranking returns naturally. If it does, this was the problem for sure.
Related posts
Tags: Duplicate, duplicate content, File, google, google ranking, homepage, htaccess, htaccess file, HTTP, keywords, link, php, Proxy, REQUEST, search results, traffic stats, URL, urls, website, website content, website diagnostics, www
Posted in SEO, Websites | No Comments »
Saturday, September 20th, 2008
As I wrote in a prvious post, duplicate content on your own website can come in the form of “www.mysite.com/” vs. “www.mysite.com/index.html.” The search engines see this same page as two different ones, but with identical content. As I also mentioned, most search engines are smart enough to figure out that these two pages are the same one, but still, they do share Pagerank.
What to do? That’s easy too. Just open up your .htaccess again and type in the following code:
RewriteEngine On
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index\.html\ HTTP/
RewriteRule ^index\.html$ http://www.mysite.com/ [R=301,L]
You can do this with other pages that have the same problem as well.
Related posts
Tags: code, content, Duplicate, duplicate content, htaccess, html, HTTP, pagerank, REQUEST, RewriteCond, RewriteEngine, RewriteRule, search engine, search engines, website, www
Posted in SEO, Websites | No Comments »
Wednesday, September 17th, 2008
Ok, this is a pretty simple thing to do and it has some important benefits.
Have you ever visited a website or a web page only to find that annoying “Not Found” error message? If so, what did you do? You probably got ticked off, hit the back button and visited another website. Can you imagine someone coming across a “Not Found” error page on your website? Well, if you don’t have a custom 404 “File Not Found” page set up on your website, that might just be happening.
Here is what you need to do to fix this problem and keep your visitors on your website.
The first thing is to create a web page with some sort of message on it. Something like, “Whoops, looks like the page you are looking for isn’t here. Please click this link to visit our home page or our search page…” You get the idea. You can save the page as “404.php” or something similar and upload it to the root of your web server.
Oh, I forgot to mention this. In order to do what I am suggesting here, you need to be running an Apache web server and your web host has to allow changes to your .htaccess file. I am sure there are other ways to create a custom 404 File Not Found error page and get it up and running, but I am only talking about one way here.
Now, open up your .htaccess file and place this code into it somewhere. I like to place it right on top:
ErrorDocument 404 /404.php
I am using .php extensions for this stuff just because of habit and preference. You can use .html or whatever you wish.
Well, that’s basically it. You can now save your .htaccess file and upload it to the server and go see if it worked. Try typing in some page that you know isn’t there. If it works, please read my previous post about “How To Check Your Web Page HTTP Headers & Response Codes” for some important information.
Good luck.
Related posts
Tags: 404 php, Apache, apache server, apache web server, code, Codes, custom, custom 404, error, error message, ErrorDocument, File, file upload, header, headers, HOST, htaccess file, html, HTTP, link, php, php extensions, Response, response code, root, search, search page, server, web host, web page, website
Posted in Websites | No Comments »
Tuesday, September 16th, 2008
There may be cases when you would like to see what your webpage HTTP headers look like. Why? Well, because they are kind of important. As Wikipedia states, the HTTP headers define what the returned data looks like.
Still you ask, “Why in the world do I care about that?” Ok, I’ll keep going. The main reason I look at the HTTP headers is to find out what the HTTP status code is. The reason the status code is important to me is because this is the code the search engines use for a multitude of things.
Let me give you a little example, and this related to my previous post regarding the sudden drop in Google rankings. As I was doing research into what the problem may be for this particular website, I came across an issue where someone had recently put custom “404 Not Found” error pages up on some of their websites. Everyone knows that custom “404 Not Found” error pages are cool, but what some people don’t know is that if those 404 error pages show a “200 OK” (successful HTTP requests) code, the site may be in big trouble, SEO-wise. The reason for this is because there are going to be many “404 Not Found” error pages on a dynamic website. If you have your custom “404 Not Found” error page showing a “200 OK” response code, the search engines will think that all the instances of this page are duplicate. You know as well as I do, that spells trouble.
What’s worse is if you set your homepage as your “404 Not Found” page. Your homepage is going to return a response code of “200 OK.” That’s not good, because now you have multiple instances of your homepage…all duplicate content.
It’s my opinion that the search engines are smart enough to figure this out. The page (such as your homepage) with the highest Pagerank will prevail. Still, I have some websites that I am working on that have multiple instances of the homepage and they all have Pagerank, which isn’t good, because the duplicates are taking the Pagerank from the real page. Now, again, that’s my opinion.
Here are two tips:
- How to check your HTTP headers - visit this website or just Google “Website header check”
- How to set a particular page as your “404 Not Found” error page in your .htaccess file - Just place this code in the file: “ErrorDocument 404 /404.php” without the quotes. The 404.php file is the actual error page in this case.
Related posts
Tags: 404 error, 404 not found error, Codes, custom, data, Duplicate, dynamic website, ErrorDocument, google, google ranking, header, headers, homepage, htaccess file, HTTP, ok response, pagerank, php, REQUEST, response code, search, search engine, search engines, SEO-wise, status, webpage, website, Websites, wikipedia
Posted in SEO, Websites | 2 Comments »
Tuesday, September 16th, 2008
Do you remember my article from yesterday about the sudden drop in Google search ranking for my friend’s website? Well, I just can’t stop thinking about it.
From what I have been reading, it seems as though my conclusion may be correct. At least I am hoping it is. If I ever conclude anything semi-concrete while thinking about Google, it’s a good day for me.
Ok, I found this very helpful and thorough website that pretty much described the exact problem my friend is having. It’s titled “Google Proxy Hijacking” and tells the whole story.
Here is what struck me as I think about this some more.
- My friend’s website has been live since 2004.
- The site seemed to be in the Google sandbox for the entire 4 years.
- For his most competitive keywords, he was ranking past page 20 on Google.
- About two months ago, he made some changes to the homepage copy as well as an HTML overhaul.
- About a month after that, the site ranked number 3 for his most competitive keywords.
- The site ranked on page 1 of Google for about a month.
- The site now sits at page 25 for its most competitive keywords.
Here is my theory. I think the website has been proxy hijacked for a number of years. This is what caused the poor rankings for such a long time. When the homepage text and HTML changes were made about 2 months ago, Google visited the site and found it unique. Google ranked the site well, due to this new unique content. During the month, Google noticed the proxy website was now a duplicate of my friend’s website once again and dropped the website’s ranking.
Does that make sense? From what I read on the website I linked to, it does.
Here are the similarities with what we are experiencing and what the author wrote on the other website:
- My friend’s website has never been banned.
- We did a quoted Google search for supposedly unique content on my friend’s website and a proxy website showed in the search results.
- The proxy URL looked like this: proxysite.com/cgi-bin/pxy/nph-pxy.pl/000010A/http/www.friendssite.com/
- The proxy site was an exact duplicate of my friend’s website.
Now, I am not sure if this is what caused my friends ranking to drop, but all the factors are there. The keywords we are talking about are very competitive, but the fact that his site showed so well in the search results for a month shows me that the potential is there.
I would appreciate your thoughts on this.
Related posts
Tags: content, Duplicate, google, google ranking, google search, Hijack, hijacking, homepage, homepage copy, homepage text, html, HTTP, keywords, Proxy, proxy url, proxysite, sandbox, search ranking, URL, website, www
Posted in SEO, Websites | 4 Comments »
Monday, September 15th, 2008
Today has been an interesting day. We have been taking a look at our websites and searching for duplicate content using Copyscape. After today’s findings, we might just go with Copyscape’s premium service.
Now, let me just tell you that duplicate content is everywhere. Actually, someone has probably written this sentence a million times. What we were searching for today was blatant and far reaching content theft. We found a few instances of one of our homepages and general website idea taken for someone else’s use as well as many instances of interior pages taken. Needless to say, we made screen copies of these cases and sent them to our attorney’s office. These are serious and can’t be ignored.
I would like to talk about two things you can do to help out a more subtle form of duplicate content, on your own website.
The first form of duplicate content on your own website is in the form of www vs. non-www. If you go to your website and type in “www.mysite.com” and then type in “mysite.com,” you may see the same page appear. In the search engine’s eyes, these are two copies of the same page. How do you fix this? It’s easy. Just open up your .htaccess file and type in the following code:
RewriteEngine On
RewriteCond %{HTTP_HOST} !^www\.mysite\.com
RewriteRule ^(.*)$ http://www.mysite.com/$1 [R=permanent,L]
When someone types in “mysite.com” to visit your website, they will automatically be forwarded to “www.mysite.com.” The search engines will be forwarded as well.
Another form of duplicate content on your own website comes in the form of “www.mysite.com/” vs. “www.mysite.com/index.html.” The search engines see this same page as two different ones. What to do? That’s easy too. Just open up your .htaccess again and type in the following code:
RewriteEngine On
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index\.html\ HTTP/
RewriteRule ^index\.html$ http://www.mysite.com/ [R=301,L]
When someone either types in “www.mysite.com/index.html” or follows a link like that to your website, they will be automatically be forwarded to “www.mysite.com.”
Now, here is the disclaimer. I used this on my server setup and it worked. Please check with your own hosting company to see if something similar will work for your too.
Related posts
Tags: code, content, content theft, copyscape, Duplicate, HOST, htaccess file, html, HTTP, link, REQUEST, RewriteCond, RewriteEngine, RewriteRule, search, search engine, search engines, server, website, Websites, www
Posted in SEO, Websites | 1 Comment »
