Doh! 404 Error!
If you find that Google is listing results for your website that are no longer valid, this may affect your search engine rankings and will certainly affect potential visitors to your website. This can arise due to a number of reasons.
- You’ve rebuilt the website using different page names
- You’ve rebuilt your website in a different language eg: changed from ASP to PHP
- Your website structure has changed
- Your domain name has changed
- You have .htaccess issues
If you have only a few pages that have changed, you could use Google’s URL Removal Tool. However, the down-side to this is that you may lose links from other websites that were pointing to the page that you had visible before your website changed. If it was a really important page, there could be hundreds of websites linking to it and every one of those mean lost visitors if the new page can’t be found.
301 Redirects
The best way to prevent lost links and visitors is to insert a 301 redirect for the altered page into an .htaccess file. This is easy to do if only a few static pages have changed. Here’s an example…
redirect 301 “www.oldsite.com/old-widgets.asp”
http://www.newsite.com/new-widgets.php
In this example, the page has been changed from ASP to PHP, from an old domain to a new domain and also from an old page name to a new page name. 301 redirects are the best way of transferring web traffic from old pages to new pages without removing the URL from Google.
However, if you had hundreds of pages that had changed, it’s not always easy to set up all the redirects manually and you may need to consider an alternative solution.
Google URL Removal and Robots.txt
One thing you definitely shouldn’t do is use Google’s URL Removal tool for lots of web pages. Spamming Google with requests like this is not a good idea!
If you do need to remove a whole website or whole directory, you can used the robot.txt file to block access from the domain or directory and then use the removal tool to request removal of it entirely. The robots.txt file will create a 404 error that Google needs to confirm that the removal request is genuine. You wouldn’t want just anyone to be able to request removal of your website would you?
Domain name changes
If you’ve registered a new domain name or bought a new domain and you want your website to use it instead of the old one, there’s an even easier way to fix link problems. For your search engine listings, you can send Google a change of domain name notice to update all your listings. The best practise once your new site is live is as follows…
- 301 redirect all traffic from the old domain to the new domain name.
- Add and verify your new domain with Google
- Via Google webmaster tools, send through the change of address request.
.htaccess issues
Occasionally, problems may be caused by someone trying to be tricky with the .htaccess file. I recently had a problem like this where the server admin had put three ecommerce websites under the same directory and used the .htaccess file to direct the domains to different folders. They did this because the way they offered hosting accounts to their clients did not allow unique access to each website. It caused many hours of work for the client, myself and the server admin every time we upgraded the client’s shop. Eventually, we had to shift the website to another server. Just something extra to look out for!